Tory MPs have been urged to accept a "compromise" customs plan proposed by No 10 or face having a "watered-down" Brexit forced on them by Parliament.
Ex-leader Lord Hague said if Brexiteers pushed Theresa May too hard, it would increase the chance of the Commons voting to stay in the customs union.
A No 10 source said the "best of both worlds" plan offered the opportunity for friction-free trade with the EU and the scope to strike deals elsewhere.
Ministers will discuss it on Friday.
The UK is due to leave the EU on 29 March 2019, and negotiations are taking place on what the future relationship between the UK and the EU will look like.
Ministers have so far failed to agree what type of customs model to pursue. The decision is seen as being key to facilitating trade and to avoiding new border checks between Northern Ireland and the Republic of Ireland, which is a member of the EU.
Friday's Cabinet meeting at Chequers will be followed by a White Paper setting out details of the UK's future relations with the EU, aimed at paving the way for an agreement in the autumn.
No details have yet been announced about how the new proposed system will work although a No 10 source has promised it will be a "significant step forward".
BBC deputy political editor John Pienaar said key to the proposal would be a single market trade deal on goods - although it would not be called a single market because that suggests close ties to the EU.
It would also include the agriculture and food sector to avoid a hard border in Northern Ireland - but not necessarily the financial services industry, he said.
On customs, the UK would apply EU tariffs except where it can secure international trade deals, he added - with "some concession" in the area of free movement of people also expected.
The prime minister's "big gamble" is whether her cabinet will buy her plan, our correspondent said.
Leading Tory Brexiteer Jacob Rees-Mogg told BBC Radio 4's The World at One that such an arrangement would be "deeply unsatisfactory".
Mr Rees-Mogg - who has been accused of "threatening" Theresa May over Brexit - said he was simply "supporting the prime minister in what she's previously said".
When is a plan not a plan?
BBC political editor Laura Kuenssberg
The immediate problem with the "new plan" is whether or not it really exists.
Because while Number 10 says it does, ask other people in government and they are not quite so sure. Ministers who you might have thought would be aware of the detail like - oh, you might imagine - the Brexit secretary had not agreed the lines, before Number 10 made their intervention.
And it's said that the reason he has not been involved in agreeing the "new plan" is because it does not actually exist yet.
The customs decision expected in some parts of government therefore is what has been anticipated for some time as "max fac plus" - a souped up version of the proposal that originally won the day in the Brexit subcommittee what feels like a lifetime ago - with, you assume, a long lead-in time while the technology is made to work.
Baffled? Quite possibly so. But it's perhaps only safe to say that four days before ministers are expected to actually make some final decisions, all is not precisely as you might have expected.
But former foreign secretary Lord Hague, a life-long Eurosceptic who campaigned to remain in 2016, has said the Brexiteers should be careful what they wish for.
In an article for the Daily Telegraph, he said they did not have the numbers in Parliament to force through a "hard Brexit" - leaving the UK without any deal over market access or long-term customs arrangements.
"If ardent Brexiteers push too hard, they will end up without their main objective," he wrote.
"If there is no agreement this week on a plan for customs arrangements, the Commons will be much more likely to vote in the near future to stay in the customs union in its entirety.
"The choice is either to back a compromise plan now or to end up with a more watered-down version of Brexit that would be forced on ministers anyhow."
The government is committed to leaving the current customs union but the two alternatives it proposed last year, a so-called customs partnership and maximum facilitation system, are believed to now be regarded as practically or politically undeliverable.
On Monday, amid signs of growing Tory divisions, Foreign Secretary Boris Johnson spoke up in defence of his "principled" colleague Mr Rees-Mogg who had been criticised by other government ministers for his Daily Telegraph article.
Brexit minister Steve Baker said Mr Rees-Mogg was right to air his concerns.
Lord Hague said the "sensible middle" in his party would not forgive anything which destabilised Mrs May's position at such a critical time in the negotiations or was seen as "naked" political manoeuvring.
"Everyone threatening Theresa May with chaos, revolt, resignations and a leadership election if she does not do as they wish needs to think carefully about what might be the consequences of their actions," he said.
The prime minister visited Dutch leader Mark Rutte on Tuesday although No 10 has insisted that no foreign leader will be given sight of the UK's latest plans before they are discussed at Cabinet.
Mr Rutte said: "We urgently need clarity about every aspect of the future relationship between the EU and UK and we are working very hard to achieve it."
Mrs May said she wanted to secure "a future relationship that reflects the close ties we have with the Netherlands".