The Conservatives have described their plans to stop British laws being overruled by human rights judgements from Strasbourg as "viable and legal".
Justice Secretary Chris Grayling said if the Tories won the 2015 election, a new Bill of Rights would give UK courts and Parliament the "final say".
There should be no "legal blank cheque to take human rights into areas where they have never applied", he added.
But former Attorney General Dominic Grieve said the plans were flawed.
The Tory MP said they would be "difficult to implement" and risked "undermining" the UK's - and his own party's - tradition of upholding human rights.
Labour and the Lib Dems have said the proposals are politically motivated while the UK Independence Party claimed they were "worthless".
The Conservatives have pledged for a decade to scrap the 1998 Human Rights Act, introduced under the Labour government, which incorporates the European Convention on Human Rights into British law.
In his speech to the Conservative conference on Wednesday, David Cameron said if his party formed the next government, it would replace the Human Rights Act with a British Bill of Rights and Responsibilities.
The Tories have also said they would be prepared to exercise their right to withdraw from the European Convention if Parliament and the British courts could not veto laws from applying to the UK.
The Council of Europe, comprising European Convention member states, said it was "inconceivable" that the UK, as a founding member, could leave.
Analysis by legal correspondent Clive Coleman
The European Convention on Human Rights was concluded in the aftermath of World War Two, drafted by British lawyers and supported by Winston Churchill.
It enshrined human rights that applied equally to all, the good guys and the bad guys.
The UK signed up to that and to a 'club', The Council of Europe, whose members work cooperatively on matters relating to human rights and the rule of law across Europe.
A condition of membership is abiding by the decisions of the European Court of Human Rights.
These proposals address what is described as mission creep, decisions of that court into areas never intended.
They amount to saying, we want to change the rules of the club as they apply to us and we want to be able to limit human rights in respect of some people who abuse their responsibilities.
The Conservatives say they'd write the text of the European Convention into UK law, but with limitations on some rights.
But critics say this is human rights "a la carte", and that it is not legally possible without the UK leaving not just the Council of Europe, but the EU.
But Mr Grayling told BBC Radio 4's Today programme that the European Court of Human Rights, whose role it is to apply the rights set out in the European Convention, had "moved a long way away" from its founders' principles.
Laws, he said, should apply in only the most serious cases and be clearly defined to stop courts applying them to "whole new areas of public life".
"I think the people of this country believe that... decisions on these things - such as whether prisoners should be sent to jail for the rest of their lives without the chance of release and whether prisoners should be given the vote... should be addressed in our courts and in our Parliament."
Under the plans, Strasbourg judgements would be treated as "advisory" rather than binding while the UK's Supreme Court would be "the ultimate arbiter" on human rights matters.
Mr Grayling added: "We have taken strong legal advice on this. The attorney general has looked at our plans and agreed they are fine, viable and legal."
But Mr Grieve - who was sacked as attorney general in July - questioned how such changes could be enforced in Scotland, Wales and Northern Ireland.
Recent ECHR rulings include:
- 2014: The latest in a series of rulings against the UK's blanket ban on prisoners having the vote
- 2013: Ruling that whole-life tariffs breach prisoners' human rights
- 2012: Blocking the deportation of radical cleric Abu Qatada from UK to Jordan
- 2010: Upholding a complaint against British anti-terror law allowing police to stop and search people without firm grounds for suspicion
"It is right to say that on a daily basis the court is producing decisions of great importance in improving human rights in Europe, which are inevitably ignored here because they tend to concern countries in Eastern Europe," he said.
While backing reform of the court, Mr Grieve said the Conservatives' proposals contained "a number of howlers" and "factual inaccuracies" about recent judgements - something that Mr Grayling disputes.
Mr Grieve later told Sky News he wished he had been consulted in advance of its publication, suggesting he had been kept in the dark because his criticism was regarded as "undesirable".
BBC political correspondent Iain Watson said the Conservatives could implement their plans only if they won an outright majority next year.
Shadow justice secretary Sadiq Khan said the European Court of Human Rights needed reform but the Human Rights Act was, to all intent and purposes, a British Bill of Rights.
Of the 1,700 cases brought against the UK government last year, he told the BBC, only eight had been upheld.
"The truth is that our courts have been free to interpret rulings by the European Convention on Human Rights for 50 years - the Human Rights Act did nothing to change that fact," he said.
"Leaving the ECHR, which the Tories appear to be proposing, would be a disaster for this country - putting Britain in the same bracket as Belarus, Europe's last remaining dictatorship."
And Lib Dem minister Simon Hughes said "hard-won" rights should not be sacrificed for short-term political reasons.
"These plans make no sense - you can't protect the human rights of Brits and pull out of the system that protects them," he said.
"Europe's human rights laws were designed by British lawyers to reflect British values of justice, tolerance and decency."
UKIP said the European Court was now "integrated" within the EU and the UK would be bound by its provisions unless it pulled out of the EU.
"This is red meat for the troops before an election, but will turn out to be thin gruel if they ever try to enact it," said its leader Nigel Farage.
The Council of Europe said the proposals were "not consistent" with the European Convention while Amnesty International said human rights "should not be dragged into Britain's internal debate on Europe".
The Law Society, which represents solicitors in the UK, said scrapping the Human Rights Act would "weaken the protection" of human rights.