The justice secretary has asked the Serious Fraud Office to consider investigating G4S over overcharging for tagging criminals in England and Wales.
Chris Grayling told MPs overcharging by G4S and rival Serco amounted to tens of millions of pounds.
He said some charges were for tagging people who were in jail or abroad, and a few who had died.
Serco agreed to a new "forensic audit" by accountants PriceWaterhouseCoopers but G4S declined, he added.
The firms have said they will repay any amount which is found to be due.
G4S said it had "co-operated fully" with a previous audit which PwC began earlier this year and was conducting its own inquiry but was not aware of any indications of dishonesty or misconduct.
Shares in the companies fell as Mr Grayling said there would also be a wider review of all contracts held by the two companies across government.
Last month, figures showed government spending on contracts with G4S had risen by more than £65m in 2012 to £394m.
In a Commons statement, Mr Grayling said he had also launched a disciplinary investigation into the way the contracts had been managed inside the Ministry of Justice after uncovering evidence that officials knew in 2008 there were problems with billing.
Electronic tagging of criminals is a key part of the government's strategy to monitor offenders in the community. Contracts are awarded to private companies who place the electronic ankle bracelet on the offender or suspect and ensure that their movements comply with their bail or licence conditions.
Mr Grayling said that current contracts had been awarded in November 2004 and were due to expire shortly.
He said that auditors had revealed a "significant anomaly in the billing practices" of both companies.
Mr Grayling said: "It included charges for people who were back in prison and had had their tags removed, people who had left the country, and those who had never been tagged in the first place but who had instead been returned to court.
"There are a small number of cases where charging continued for a period when the subject was known to have died.
"In some instances, charging continued for a period of many months and indeed years after active monitoring had ceased."
'Open and transparent'
Mr Grayling said officials estimated that the total over-billing was in the "low tens of millions".
He said that Serco had agreed to the forensic audit [that is, a search for possible illegality] of what happened, which included looking at executives' emails.
Mr Grayling said: "We put the same proposal for a further detailed forensic audit to take place to G4S. They have rejected that proposal.
"I should state that I have no information to confirm that dishonesty has taken place on the part of either supplier.
But following the findings of the previous audit and after receiving legal advice "I am today asking the Serious Fraud Office to consider whether an investigation is appropriate into what happened in G4S."
Shadow justice secretary Sadiq Khan said the government should ask the police and the Serious Fraud Office to investigate both G4S and Serco.
"Both these companies are recipients of hundreds of millions of pounds of contracts from across government and local authorities," he said.
He also said plans to privatise probation contracts now needed to be evaluated through pilot schemes.
The Ministry of Justice said G4S co-operated with the first audit. It added that the company did not receive the full results, but was given the categories of over-charging and examples. Serco received the same information. The ministry has not commented on suggestions that G4S warned the MOJ in 2009 about anomalies in the charging arrangements.
In a statement, the company said it believed that "any evidence or indication of dishonesty should be referred to the relevant authorities including, if appropriate, the SFO".
G4S added: "We can confirm that we are working with the Ministry of Justice on their review of the electronic monitoring contracts.
"We believe that we are delivering our electronic monitoring service in a completely open and transparent way."
G4S chief executive Ashley Almanz said: "We take very seriously the concerns expressed by the Ministry of Justice. We are determined to deal with these issues in a prompt and appropriate manner."
Serco's chief executive Christopher Hyman said: "We are deeply concerned if we fall short of the standards expected of all of us.
"We are therefore taking this extremely seriously and will continue to work closely with our customer to resolve their concerns in this matter. We will not tolerate poor practice and behaviour and wherever it is found we will put it right."
Serco has withdrawn from the tendering process for the next generation of tags.
Ian Lawrence, general secretary of the National Association of Probation Officers, said Serco and G4S should not be allowed to bid for any of the contracts under proposals to privatise the probation service
Andrew Neilson, director of campaigns at the Howard League for Penal Reform, said Mr Grayling "should rethink his plans to introduce yet more private sector involvement in the criminal justice system".