Justice Secretary Ken Clarke has warned MPs against voting down plans to allow some prisoners the right to vote.
He said ministers had to obey the law - the European Court of Human Rights ruled in 2005 the UK's blanket ban on prisoner voting was unlawful.
Mr Clarke told the BBC there had been some "over-excitement" about the story and most prisoners would not bother to vote anyway.
Next week MPs will debate the plans. Many Tory MPs have threatened to rebel.
The debate was secured by senior Conservative backbencher David Davis and former Labour cabinet minister Jack Straw who convinced a new Commons committee to allow a vote on the issue.
Mr Clarke said nobody wanted to give lots of prisoners the right to vote, but the government had to obey the rule of law.
Asked what the government would do if it lost the vote, Mr Clarke told BBC Radio 4's World at One: "I shall ask anybody who thinks they are going to drive to that particular point how they are going to explain to their constituents that, at a time like this, we're spending money on compensating prisoners for a right that they probably wouldn't bother to exercise if we gave it to them.
"I wish to get out of this with the minimum damage and I think there's a certain amount of over-excitement surrounding it. I would have left things as they were. We shall do the minimum that we have to to comply with judgements against us."
The row over prisoners voting in the UK has dragged on since 2005 after the European Court of Human Rights said that the UK's blanket ban was unlawful.
In June the Council of Europe, an inter-governmental organisation that oversees and enforces rulings made by the ECHR, urged the coalition to act.
The government argues it has been advised that, unless the law is changed, it could face compensation claims from prisoners costing well over £100m. It has proposed giving the vote to those sentenced to a year or less.
But many Conservative MPs are annoyed, not only at the idea of giving the vote to prisoners but at the power of the ECHR.
Mr Davis, a former shadow home secretary and Tory leadership contender, said a "crisis" had been created "by the court extending its own power, trying to overrule, in effect, a parliament".
And Mr Straw, who delayed responding to the ruling when he was the responsible minister under Gordon Brown, has denied trying to cause trouble for the government. He said the ECHR had argued there had been no substantive debate on the policy - and he was calling for that debate now.
On Tuesday, lawyers giving evidence to a Commons committee suggested the right would have to be extended to give prisoners in Scotland and Wales the right to vote in May's elections - ministers had thought it only applied to Westminster and European Parliament elections.
Asked whether prisoners would get to vote in the devolved elections, Mr Clarke said the government would "have to see what the courts decide".
"Our proposal is not to, but we'll have to see what the adjudication is... let's wait for the outcome. What we have to do is do the minimum necessary to comply with our legal obligations."
Asked about next week's debate in the Commons, he said: "I hope people will take on board that we do have the rule of law here, what the government actually proposes has to be consistent with the law. Nobody wants to give large numbers of prisoners the vote so we will do the minimum necessary to avoid further chaos.
"What we don't want is more and more prisoners getting more and more legal aid and starting to be paid compensation because the House of Commons won't bring itself to face up to what every disappointed litigant has to do - that it's got to do what the courts have ordered it to do."