Ministers have been criticised by the UK spending watchdog for failing to deport more foreign criminals.
The National Audit Office said the number of foreign prisoners had gone up despite a near tenfold increase in officials working on their cases.
There are 12,250 foreign offenders in the UK and the government spent £850m last year on managing them.
The Home Office said recently enacted powers meant it was becoming easier to remove foreign offenders from the UK.
Removing foreign national offenders (FNOs) became a Home Office priority in 2006 following the sacking of the then Home Secretary Charles Clarke.
He lost his job after it emerged that 1,000 foreign inmates had been released without being considered for deportation.
'High harm' individuals
But the NAO's report found that as of March 2014 performance was broadly unchanged. There were 10,650 prisoners from overseas - slightly more than in 2006.
Despite an initial surge in removals to more than 5,600 in 2008-09, the number leaving the UK had fallen back to 5,100 in the last financial year.
Officials estimate that between January 2009 and March 2014, 151 FNOs left prison without being considered for deportation. One in six FNOs in the community - 760 convicted criminals - had absconded, said officials.
Almost 400 of them had been missing since before 2010, of which 58 were classed as "high harm" individuals.
Amyas Morse, head of the National Audit Office, said: "It is no easy matter to manage foreign national offenders in the UK and to deport those who have completed their sentences.
"However, too little progress has been made, despite the increased resources and effort devoted to this problem."
Auditors analysed almost 1,500 failed removals and found that a third might have been avoided had there been fewer administrative errors and better co-ordination between different parts of government. Only one in every 25 files sent to the Home Office to start the process for removal had the right identity documents.
Paul Houston's 12 year old daughter, Amy, was knocked down and killed in in a hit-and-run incident in Blackburn, Lancashire, in 2003, by an Iraqi Kurd who was banned from driving and had been refused asylum.
He is still in the UK and Mr Houston told BBC Radio 4's Today programme that politicians had "blood on their hands".
"They owe it to society to protect us from dangerous criminals," he said, adding that a "lot of money has been spent" for "very little progress".
'Affront to justice'
Immigration and security minister James Brokenshire said that foreign national offenders should be in "no doubt" of the government's resolute approach to removing them.
He said the government had removed about 22,000 foreign criminals against a backdrop of an increasing number of appeals.
"The countless appeals and re-appeals lodged by criminals attempting to cheat the system cost us all money and are an affront to British justice. That is why we are putting a stop to that abuse through the Immigration Act.
"New powers came into force this week to cut the number of grounds on which criminals can appeal their deportation and to end the appeals conveyor belt in the courts."
He told the BBC: "We can, we must and we will do more to see that foreign national offenders are speeded through the process through ensuring that we have the right court system in place. That is what we're now doing."
But shadow immigration minister David Hanson said the prime minister had failed to deliver on his personal commitment to deport more FNOs.
"This report shows the Home Office is simply not doing enough to crack down on removing foreign prisoners which would save the taxpayer £70m each year," he said.
"The last Labour government made this a priority and increased the number of foreign prisoners removed. How can it be that after five years in office David Cameron's government are deporting over 500 fewer people than in 2010?"