The Scottish Secretary says he would back the idea of English devolution.
Alistair Carmichael said it was the "unfinished business" of constitutional reform that has taken place since the Scottish Parliament was set up.
The Lib Dem MP for Orkney and Shetland said new powers for Holyrood were being promised and it was an "anomaly" that he could vote on England-only policies.
He said there was a debate to be had in England on an English parliament or regional assemblies.
Speaking to John Pienaar on BBC Radio 5 live, Mr Carmichael said: "Federalism is the logical conclusion of all this - but that is the decision for people in England to make."
Voters in Scotland will go to the polls on 18 September to decide on their country's future in the independence referendum. The Scottish National Party say the only way to guarantee greater powers for Holyrood is to vote yes. It has dismissed promises from the pro-union Westminster parties of new tax-raising powers if voters reject independence.
Mr Carmichael told the BBC: "We had a 20-year debate before we were able to decide properly what we wanted in Scotland. We had a campaign for a Scottish assembly; we then had a constitutional convention."
He said: "Coming on now for 17 years after it was first set up in an Act of Parliament, we've still not finished the process of devolution because there will be more tax raising powers coming to the Scottish parliament and that's a very necessary thing to rebalance the political debate in Scotland."
He added: "The logic of the position would be that if the English sort out their constitutional position.
"It is for the English to do that - do you want an English parliament, do you want English regional assemblies, that's a debate that you have to have in England for yourselves... I don't particularly want to vote on things that are not directly affecting me."
His comments come after former Wales Secretary John Redwood raised the issue in a blog entry earlier this month.
"Scottish members of the Westminster Parliament come to debate the Union matters, but cannot contribute to health, education, local government or criminal justice in their country," the Tory MP wrote. "This lopsided devolution cannot survive a further major increase in powers of the Scottish Parliament."
MPs from English constituencies have an overwhelming majority in the House of Commons. But critics say successive governments have failed to deal with the so-called "West Lothian question" - whereby Scottish, Welsh and Northern Ireland MPs can still vote on legislation primarily affecting England while English MPs have no input into devolved matters.
Former Deputy Prime Minister John Prescott championed regional assemblies for England in Labour's 2001 election manifesto but his proposals were abandoned after voters in the North East rejected the idea in a referendum in 2004.
As well as an English Parliament or regional assemblies, there has been a suggestion that some key stages of legislative scrutiny of bills affecting England should be restricted to English MPs. Most recently, the principle was backed by the McKay Commission taskforce set up by the coalition government.