Private security firm Serco has agreed to repay about £68.5m to the government for overcharging on contracts to tag criminals in the UK, Justice Secretary Chris Grayling has said.
It follows allegations Serco and G4S charged the government for tagging people who were either dead or in jail.
The Ministry of Justice also says there were "serious issues" in two G4S contracts to manage court facilities.
Serco said it "apologised unreservedly" for the contract issues.
The matter concerning the G4S contracts has now been referred to the Serious Fraud Office (SFO).
An SFO spokesman said: "We are aware of the Ministry of Justice's referral and will urgently consider the information they pass to us."
Both security firms have withdrawn from the bidding process to win contracts to supervise offenders on their release from prison.
Earlier this month Mr Grayling said outsourcing company Capita would take on the tagging contracts by next April.
An audit by accountants PricewaterhouseCoopers alleged that overcharging began as early as 2005.
Meanwhile, a cross-government review of G4S and Serco contracts found weaknesses in the way the majority of contracts were managed, some of which were significant.
BBC political correspondent Ross Hawkins said the review had uncovered three potentially significant problems in Department for Work and Pensions contracts for running the government's Work Programme.
He said Cabinet Office Minister Francis Maude had insisted the government would stick with outsourcing.
"The picture at the moment isn't pretty," our correspondent added. "They say they can sort it out but they've got a lot of work to do."
Alastair Lyons, non-executive chairman of Serco, said: "The contract issues that were identified should never have happened and we apologise unreservedly for them.
'Concerns taken seriously'
"We are doing everything in our power to make sure that such issues cannot reoccur anywhere in our business around the world.
"Our objective is to deliver excellent public services with openness and transparency, and I believe the actions we are taking will support this now more than ever."
A statement from G4S said the company "places the highest premium on adherence to its company values, including customer service and integrity.
"We take the MoJ's concerns very seriously and the company has recently strengthened the service management team on the court FM contracts and continues to work closely with the MoJ to ensure that G4S delivers a facilities management service to the courts that reflects the high standards of performance which G4S expects to provide to all customers."