David Cameron's "impossible" pledge to reduce net migration to below 100,000 must be abandoned if he wins the referendum, Vote Leave has said.
Gisela Stuart, who chairs the group, said promising to control numbers and then failing to do so was "corrosive" of public trust in politicians.
Voting Leave on Thursday was the only way to "take back control" on immigration, she said.
Mr Cameron said leaving the EU was "not the right way to control immigration".
Meanwhile, a report by think tank Migration Watch, that wants lower immigration, said overall, all migrants - both from the EU and elsewhere - in the UK "continue to be a net fiscal cost" to the Exchequer.
It said only recent migrants - who arrived from 2001 - from the old member states of the EU14 had made a "net positive fiscal contribution".
"Immigration has not been shown to have any significant impact, either positive or negative, on GDP per capita, a key measure of economic performance.
"There is therefore no economic case for mass immigration on the present scale," the report said.
Ms Stuart said the UK was an "open and tolerant nation" that had been made "better and stronger" by immigration and should be "proud" about helping refugees.
But the Labour MP said the EU's policies were "failing in humanitarian terms".
"The tragic scenes unfolding in the Mediterranean underline how badly the European Union is handling population movements and migration pressures," she said.
"If we are to maintain public confidence in migration and prevent extremists from dominating the debate, it is essential that elected politicians are accountable for decisions about migration."
Ms Stuart said voting to Remain meant there would be "no control" over migration from the EU, "no matter how great the pressure on schools, hospitals and housing becomes or how much wages in our poorest communities are pushed down".
She pointed to the fact that in 2015 net migration - the difference between the number of people coming to the UK for at least a year and those leaving - was 333,000, according to the Office for National Statistics.
That figure was the second highest on record, while the figure for EU-only net migration was 184,000 - equalling a record high.
Ms Stuart said: "It is clear that it will be impossible to reduce net migration below 100,000 if we vote to stay in the EU.
"The prime minister must now make clear that he will abandon this manifesto pledge if he wins the referendum on Thursday. He cannot continue to promise to do something that he knows is impossible.
"Inside the EU we don't control our borders and cannot control the movement of people coming here from the EU."
Ms Stuart said that by leaving the EU the UK would "take back control" and politicians would have to keep their promises on migration.
She said in the past Mr Cameron had ended up dropping promised policies on EU migration - such as EU migrants having to have a job offer to come to in the UK - "because Brussels would not allow them".
In a special edition of Question Time, Mr Cameron conceded that controlling immigration was "very challenging" but he said his EU renegotiation would make a "big difference".
He said there was "no silver bullet" and that leaving the EU and the single market was "not the right way to control immigration".
The PM also said the government would have to put up taxes or cut spending or risk "going back to square one" if the UK voted to leave.
If the UK votes to leave, he said: "That's it, we are walking out the door, we are quitting - we are giving up on this organisation."
"I do not think Britain, at the end, is a quitter," he added.
Speaking on BBC Radio 4's Today programme on Monday, UKIP leader Nigel Farage said he wanted to see UK immigration come down to "post war" levels of between 30,000 and 50,000 people a year.
"At the moment, we have no control. A British passport is an EU passport, open to 500 million people," he said, but added that it would be up to Parliament to set the level.
'Take back control'
Immigration has been a key battleground in the referendum campaign ahead of the vote on EU membership on 23 June.
On Sunday Labour leader Jeremy Corbyn, who supports Remain, told the BBC there could be no upper limit on the number of people coming into the UK while there is free movement of labour in the EU.
Pro-Leave Michael Gove said people wanted to feel numbers were controlled.
Meanwhile, Boris Johnson, another leading Leave campaigner, said that without border controls, "the substantial flows of migration.. show no sign of diminishing".
Writing in the Daily Telegraph, Mr Johnson said that by leaving the EU the UK would "take back control" of £10.6bn net per year, which he said would enable the country to "take back control of our borders, and install an Australian-style points-based system that is fair both to people coming from the EU and from non-EU countries".
Mr Johnson urged voters to "change the whole course of European history" by voting to leave, which he said would be "overwhelmingly positive".
"Now is the time to believe in ourselves, and in what Britain can do, and to remember that we always do best when we believe in ourselves," he added.