EU-exit ministers complaining about lack of access to official documents should "stop whingeing", ex-Labour minister Lord Mandelson says.
The Britain Stronger in Europe director said ministers opposing the government were fortunate not to be sacked.
Critics say the ban on Leave ministers seeing documents relating to the referendum is "unconstitutional".
Out campaigners said Lord Mandelson had previously been proved wrong in calling for the UK to join the euro.
Cabinet Secretary Sir Jeremy Heywood faces a grilling from MPs later on the rules under which ministers can campaign for either side in the 23 June in-out referendum.
Sir Jeremy, who appears before the Commons Public Administration Committee later, has issued guidance saying ministers opposing the official government line should not be given access to government papers on the referendum or the PM's EU renegotiations - apart from ones they had already seen.
The decision has been criticised by Conservative backbench MPs and pro-EU exit ministers who have said it risks jeopardising the political impartiality of the civil service.
Mr Cameron says the row has "got slightly out of proportion" and the public should focus on the bigger issue of the merits of staying in or leaving.
In a speech in London, Lord Mandelson said the one paper anti-EU ministers could have been expected to get was "their P45".
The Britain Stronger In Europe board member also attacked the case to leave the EU as a "fantasy".
The UK would not be in a position to negotiate a preferential post-exit EU trade deal, he said.
The former EU trade commissioner said: "Brexiters cannot argue that we are weakened in the EU as it is but would suddenly be strong enough to dictate terms if we left.
"For every politician who saw the pragmatic case for dealing with the UK, there would be another who had little doubt that the UK must not be given a quick or easy ride."
He added: "In return for market access, we would be required to continue to accept many EU norms and standards.
"As a result, we would have left the EU in order to assert our national sovereignty only to find that, as a condition of access, we did not have independence from EU regulation after all."
Lord Mandelson claimed a vote to leave would lead to "years of uncertainty" and, "in the worst case scenario, a return to paying EU tariffs" while a final free trade agreement was negotiated.
He said tariffs of up to 20% "or sometimes even more" could be imposed on UK exports, including cars and whisky.
The former Labour minister's claims were dismissed by Matthew Elliott, chief executive of the Vote Leave campaign.
"Peter Mandelson told us the British economy would fall off a cliff if we didn't join the euro and now he is indulging in the same scaremongering about the referendum," he said.
"He was wrong then and he is wrong now. He is starting to resemble a man wearing a sign saying the 'end is nigh'."
Labour MP Gisela Stuart, who is part of Vote Leave, told BBC Radio 4's Today programme her party had "stopped thinking about what the European Union really means".
The MP for Birmingham Edgbaston said the "political stability" first offered by European nations joining together was being "seriously questioned" adding that "the European Union as an institution is not responding to it properly".
In the Commons, Chancellor George Osborne told MPs the Treasury would publish "a comprehensive analysis" of the costs and benefits of Britain's EU membership ahead of the referendum.
He said a UK exit from the EU would be "a long, costly, messy divorce".