The government has announced that prisoners serving fewer than four years will be eligible to vote.
The Cabinet Office statement said all offenders sentenced to four years or more would automatically be barred from registering to vote.
The decision comes after a European court ruling which the government is obliged to implement.
Shadow justice secretary Sadiq Khan said law-abiding citizens would be "concerned" by the changes.
It comes on the same day that a man serving life for raping and murdering his seven-year-old niece lost an appeal over his right to vote.
Three judges unanimously dismissed the Court of Appeal hearing of 55-year-old Peter Chester - who is serving life for raping and strangling Donna Marie Gillbanks in Blackpool in 1977 - and refused him permission to go to the Supreme Court.
His legal team had argued denying him a vote was a "disproportionate" reaction and violated his human rights.
Five years ago, the European Court of Human Rights (ECHR) ruled the UK's long-standing voting ban was unlawful.
In a case brought by convicted killer John Hirst, of Hull, it ruled a "blanket ban" to be discriminatory.
Mark Harper, minister for political and constitutional reform, said the changes were "not a choice, it is a legal obligation".
He said: "We are ensuring the most serious offenders will continue to be barred from voting.
"If the government failed to implement this judgement, we would not only be in breach of our international obligations but could be risking taxpayers' money in paying out compensation claims."
The sentencing judge will be able to remove the right to vote from some prisoners sentenced to fewer than four years, he added.
Mr Khan said nearly 29,000 prisoners serving sentences of four years or less would be given the vote as a result of the government's decision - including those who had committed crimes such as domestic violence, burglary, wounding and assault.
"Law-abiding citizens will be concerned that serious and violent offenders will have a say in who runs this country," he said.
But the director of the Prison Reform Trust, Juliet Lyon, said: "Enfranchising prisoners serving sentences of under four years is an important step in the right direction.
"However, it does not appear to meet the requirements of European Court judgements which state that the vast majority of prisoners should be able to vote."
Under the ECHR ruling each country can decide which offences should carry restrictions to voting rights.
Prisoners on remand awaiting trial, fine defaulters and people jailed for contempt of court are already permitted to vote but more than 70,000 prisoners currently serving sentences in UK jails are prevented.
Inmates were originally denied the right to vote under the 1870 Forfeiture Act and the ban was retained in the Representation of the People Act of 1983.
The BBC's legal affairs analyst Clive Coleman said the proposed changes to the 140-year ban would be a "hot issue" when it was debated in Parliament next year.
"The government had to come up with a legislative regime that did away with the blanket ban but also was going to in some way satisfy public opinion.
"It is a very controversial issue - why four years, why that cut off point?
"Four years has always been an almost mythical period to distinguish between very serious and less serious offences, long-term and short-term offences.
"But for instance, rapists who are convicted of a rape where there are not a lot of aggravating features to the rape may be in prison for less than four years, so they may potentially have the vote," he said.