Theresa May's plan for future relations with the EU does not go far enough to protect Scotland "from the damage of Brexit", Mike Russell has warned.
The proposals were agreed after a marathon meeting at Chequers on Friday.
Scotland's constitutional relations secretary said they were a "step in the right direction".
He also suggested that if most Scots voted again to stay in the EU then that could be "recognition that Scotland wished to be independent".
After a 12-hour meeting the UK cabinet approved a document setting out the UK's vision for its future trading arrangements with the EU after it leaves in March 2019.
Under the proposals, yet to be presented to the EU, there would be a free trade area for industrial and agricultural goods, based on a "common rule book" and a "combined customs territory".
Several Tory MPs have expressed unease at the plans and UK Foreign Secretary Boris Johnson is understood to have strongly criticised the proposals during the cabinet meeting at Chequers before signing up to it.
Speaking on the BBC's Sunday Politics Scotland programme, Mr Russell questioned whether the proposals would be acceptable to remaining EU member states and to hardline Brexiteers within the Conservative Party.
Earlier, First Minister Nicola Sturgeon tweeted that "it hasn't taken long for the #chequers plan to start to unravel".
On Saturday, she had declared the UK government's agreement could be "game on" for those in favour of a single market/customs union outcome.
Mr Russell described the plans as being a "step towards" continuing membership of the EU single market and customs union - the Scottish government's preferred position.
However, he said the two issues with the current deal were "will it be accepted" by the EU and the "great deal of muttering" about the proposals from the "extreme Brexiteers" in the Tory Party and others.
He said: "We may be on a journey, it may not be a journey however that Theresa May is capable of taking, given her party."
The constitutional relations secretary continued: "Without staying in the EU there will be damage, but actually the single market and the customs union - the full single market and the customs union, the Norway option - is the least damaging part of it.
"Without freedom of movement, real freedom of movement, there are whole sections of the Scottish economy that are going to suffer very badly.
"This isn't the end point, I'm quite sure it isn't the end point. If it is a part of a journey then it is to be welcomed but that journey has a fair bit to go before Scotland will be protected in any real way from the damage of Brexit."
Scottish Conservative deputy leader Jackson Carlaw accused the first minister and her party of "undermining Scotland's best chance of a deal with the EU".
He said: "Having said Theresa May could never get her cabinet to agree a position to deliver a pragmatic Brexit, Nicola Sturgeon now finds herself outflanked by a united UK government negotiating position and she just can't stand it."
Speaking on the Sunday Politics Scotland programme, Scottish Conservative MP Andrew Bowie said he had seen a three page summary of what was agreed at Chequers but was awaiting the publication of the full white paper later this week.
"I am very happy with what I've seen," he said.
"I believe it delivers a Brexit that is not just good for business, good for jobs and good for the economy but certainly good for Scotland."
Paul Sweeney, Scottish Labour MP, said the deal agreed by the UK cabinet "falls far short of what Labour's six tests have been throughout this process".
He added: "We believe in having a comprehensive customs union and this deal does not meet this test."
He said he thought it had been "designed to manufacture a truce within the cabinet" and "won't meet the test of parliament or the test of the EU negotiators".
'Status quo gone'
Mr Russell was also asked about the SNP's stance on having a second referendum on membership of the EU.
He said: "We believe there should be a chance for the people of Scotland to say what they think of the Brexit position but that can't be a repeat of two years ago...Scotland vote one way and the rest of the UK vote the other."
Asked by presenter Gordon Brewer what would happen if the majority of Scots voted again to stay in the EU and the rest of the UK voted to leave, Mr Russell said: "That has to be laid out before the vote takes place.
"Quite clearly in those circumstances it would be a question - would that be a recognition that Scotland wished to be independent? What it would not do is confirm a status quo because there is no status quo - it's gone, it's finished."