It's one thing disagreeing with colleagues in private.
It's another, publicly describing their comments as "very unwise".
Prime Minister David Cameron says he wants "good immigration, not mass immigration".
But his cabinet colleague Vince Cable told me that his words were "very unwise" and that the PM risked inflaming extremism.
He also distanced himself from the prime minister's promise to limit immigration to tens of thousands, not hundreds of thousands, saying that was Conservative party policy, not a coalition plan.
Yes, the coalition agreement outlined a cap on immigration that the business secretary, Home Secretary Theresa May, as well as Mr Cameron and deputy prime minister Nick Clegg, worked hard to agree.
It did not have the promise of cutting immigration to under 100,000.
And agreeing the government's actual policy was not an easy process.
Clash of ideas
Vince Cable was outspoken then, calling continually for the cap to have the flexibility that business requires, while others were more keen on calming public concerns about immigration.
Indeed in the early days of the coalition that was one of the first tests of how different views could be aired in public in way that just couldn't happen under single party government.
Under a single party government they would have been portrayed as an enormous cabinet rift. Was it really that under a coalition government?
Today though, it's not really a clash of ideas that's the problem.
There is a sense in some Lib Dem quarters that the two parties had a truce on immigration, having negotiated the policy.
And that Mr Cameron has pushed that to its limit, making Mr Cable's criticism fair game.
And it is the kind of language the prime minister chose to use that has angered Mr Cable.
Perhaps if he had seen the whole of the PM's speech, like his boss Nick Clegg, rather than reading about it in this morning's papers he would not have felt the need to speak out.
A fear of being excluded can have a powerful effect.
And just as Nick Clegg's team say this speech is a Conservative prime minister speaking to Conservative voters in the middle of an election, Vince Cable and the Lib Dems are also of course fighting a campaign, speaking to Lib Dem voters.
And - as many Lib Dems have for months been concerned about maintaining a separate identity to the Conservatives - perhaps Mr Cable's comments are an attempt to do just that.
But who actually gains from his intervention?
The PM has chosen to give a strongly worded speech in the run up to an election, well aware that his stand will appeal to many members of the public, hoping to put him on the side of the majority of public opinion.
A senior Lib Dem then wades in questioning the wisdom of such an approach.
So although Mr Cable's risked a big cabinet row, David Cameron might not be entirely distressed by today's events.