Unregulated free movement of people after Brexit would "not keep faith" with the EU referendum result, the international trade secretary has said.
In a Sunday Times interview, Liam Fox said he had "not been party" to widely-reported talks about a "transitional" period when free movement is retained.
Chancellor Philip Hammond has said "it will be some time" before full migration controls can be introduced.
Mr Fox said border controls had been a key element of the Brexit referendum.
He added: "Unregulated free movement would seem to me not to keep faith with that decision.
"I am very happy to discuss whatever transitional arrangements and whatever implementation agreement we might want, but that has to be an agreement by the cabinet.
"It can't just be made by an individual or any group within the cabinet."
On that basis, Mr Fox cast doubt on claims that the free movement of EU citizens would remain in place for up to three years after the UK leaves the EU as part of a Brexit transition deal.
"If there have been discussions on that, I have not been party to them," he said.
"I have not been involved in any discussion on that, nor have I signified my agreement to anything like that."
On Friday, Mr Hammond said free movement would end in name only in 2019.
The chancellor said "many things will look similar" the day after Brexit.
"It will be some time before we are able to introduce full migration controls between the UK and the European Union," he added.
Mr Hammond said any transitional deal in the period after Brexit must end by June 2022, the time of the next general election.
But he said there must be "business as usual, life as normal" for Britons as the UK left the EU.
BBC political correspondent Chris Mason says Mr Fox's interview lays bare division in the cabinet.
Our correspondent added: "The government's negotiations with Brussels over Brexit are yet to touch on any transitional period after March 2019.
"Senior ministers appear united about what they see as the desirability and necessity of such an arrangement, but, crucially, not about exactly how it might look."