Justice Secretary Chris Grayling has said he is "absolutely certain" the Conservatives will go into the next election with a "plan" that could involve a change in human rights law.
Asked if this could mean withdrawing from the European Convention on Human Rights (ECHR), Mr Grayling told the BBC he had "not ruled anything out".
The Lib Dems said they were "committed to keeping the Convention".
The Human Rights Act incorporates the convention into British law.
Meanwhile, the Mail on Sunday reported Home Secretary Theresa May is planning to pull Britain out of the ECHR, stripping the Strasbourg court of its right to be involved in UK cases.
In an interview with BBC Radio 5 Live, Mr Grayling alluded to the possibility of scrapping the Human Rights Act - which the Lib Dems are committed to defending - saying: "Well, self-evidently, if we are going to create a new legal framework we are not going to be able to keep the framework we've got at the moment.
"We've got a lot of work to do on the detail - [to] look at exactly how we best do that, how we deliver change."
Commenting on the Convention itself, the cabinet minister said: "If anyone sits down and reads it as a document they would struggle to find a word they disagreed with. It is sensible balance of rights and responsibilities."
But, in the interview with the Pienaar's Politics programme, he added: "The decision-making in the European Court of Human Rights in Strasbourg has moved, I think, that Convention further and further away from the original intentions of its authors."
Asked to respond to critics who have claimed that being a signatory to the Convention is proof that Britain is a modern, compassionate society, Mr Grayling said: "I was a human rights campaigner. I chaired the Amnesty International Group when I was at university."
He stressed his view that human rights were "about some of the appalling things happening around the world - people being brutalised for their political views, people being put in jail".
Responding to Mr Grayling's comments, Liberal Democrat deputy leader Simon Hughes told the BBC the ECHR was a "benchmark of human rights around the world".
He said the justice secretary was clearly taking "a very right wing position" which pandered to "an obvious anti-Europeanism".
Mr Hughes added that while some structural problems had built up within the system, "the Lib Dems were committed to keeping the Convention".
A Lib Dem source also told the BBC: "There is absolutely zero prospect of Britain leaving the European Convention on Human Rights while the Liberal Democrats are in government.
"The Tories can posture all that they want about what's in their manifesto next time around, but that is up to them."
Failing to connect
Earlier, Mr Grayling was quoted in the Sunday Telegraph as saying he would expect a future majority Conservative government to scrap the Human Rights Act - which was introduced under the last Labour government.
The cabinet minister's remarks come after an outcry among some Conservatives that the party is failing to connect with traditional blue-collar members - following its defeat in the Eastleigh by-election on Thursday.
The Conservatives finished third in Eastleigh, in Hampshire - behind the Liberal Democrats and the UK Independence Party (UKIP).
Conservative party vice chairman Michael Fabricant said the party had struggled to present a distinctive message, while UKIP managed to attract Tory voters on issues such as crime and immigration.
Writing in the Sunday Telegraph, Prime Minister David Cameron responded to the warnings by insisting there would be no "lurch to the right" by the Conservatives.