Theresa May has been warned of "dire" electoral consequences for the Tories if she persists with her Brexit plan.
Former minister David Davis said it would be "obvious" to voters if the UK ended up as a "rule-taker" and not able to exploit the benefits of leaving.
The PM says her so-called Chequers plan for close links to the EU is the only alternative to leaving without a deal.
But cabinet minister Penny Mordaunt has declined to give this blueprint for EU relations her explicit backing.
The international development secretary said the prime minister had her full support, but then added it was not clear where the process would "end up".
Talks between the two sides are entering a critical phase in the hope that an agreement can be struck by the middle of November at the latest.
The BBC's Brussels reporter Adam Fleming said negotiations were intense and fluid but there was said to be a sense of optimism and movement.
After Tuesday's weekly cabinet meeting, Downing Street made clear Mrs May would not agree to any withdrawal deal including the £39bn so-called divorce bill to be paid to the EU, without a "precise" declaration of the two sides' future trade and security partnership.
Later, Brexit Secretary Dominic Raab faced MPs' questions on developments since last month's Salzburg summit, when EU leaders rebuffed the PM's plan for a free trade zone for manufactured and agricultural goods, underpinned by a common rule book, and a combined customs territory.
Mr Raab said the two sides were "closing in on workable solutions" on key issues, and stressed the need to "hold our nerve" and "stay resolute" in order to secure a good deal.
On the thorny issue of avoiding new border checks in Northern Ireland, he reiterated that the government would not accept anything that threatened the "constitutional and economic integrity of the UK".
And he said the sort of free trade agreement being demanded by some Tory Brexiteers was a "shortcut to no deal" because it would not solve the Irish border question.
Labour said rather than Mr Raab, Mrs May should be answering questions on "what went wrong" in Salzburg.
And there was a warning from the Democratic Unionist Party's Nigel Dodds, who said his party - which gives the government support in key votes - would not accept anything that "separates" Northern Ireland from the rest of the UK.
Mr Dodds warned against repeating the "debacle" when the DUP rejected a draft deal struck between the UK and the EU.
Mr Davis, who quit as Brexit secretary in July in protest at the Chequers plan - which is named after the country house where the cabinet discussed them - said the UK faced being forced into further concessions, rendering the outcome all the more unpalatable.
"If we stay on our current trajectory, we will go into the next election with the government having delivered none of the benefits of Brexit, with the country having been reduced to a rule taker from Brussels, " he wrote in a letter to Conservative MPs.
"This will not be a technicality. It will be very obvious to the electorate. The electoral consequences could be dire."
Mr Davis and other backbench critics of the PM, including former ministers Boris Johnson and Steve Baker, want a looser arrangement modelled on Canada's trade deal with the EU.
Former chief whip Mark Harper, previously seen as a loyalist, has added his voice to those calling for the PM to listen to MPs' concerns and back the Canada option.
Up to 40 Tory MPs could oppose the PM in a vote on the deal later this autumn, Mr Baker told the BBC's Politics Live, adding there could be further ministerial resignations before then if there is no U-turn.
A Canada-style deal is unviable, Mrs May has said, because it does not provide a solution to the Irish border issue and risks separating Northern Ireland from the rest of the UK.
Ms Mordaunt, a prominent Leave campaigner in the 2016 referendum, said the prime minister had her "full support" at a critical moment in the negotiations but added "we don't know where this is going to end up".
"I don't doubt her commitment and I don't doubt for one moment her understanding that we have to deliver a good Brexit, we have to honour that result," she told journalists following a speech in London on aid policy.
The EU and UK are at loggerheads over the so-called Irish backstop, the arrangement designed to guarantee that there will not be a physical border on the island of Ireland, whatever the wider UK-EU deal.
Brussels wants Northern Ireland to effectively remain in the customs union and single market - an outcome the UK has said it cannot accept as it would effectively create a border between Northern Ireland the rest of Great Britain.
The UK's counter-proposal would see the country as a whole remaining aligned with the EU customs union for a limited time after the end of 2020, when the proposed transition period ends.
The leader of the Democratic Unionists has reiterated that she would not support any Brexit deal that could lead to new economic barriers between Northern Ireland and the rest of the UK.
Speaking after meeting EU negotiator Michel Barnier in Brussels, Arlene Foster said she needed to see the "legal text" of any amended backstop proposal before deciding whether it was acceptable.