David Cameron has dismissed criticism of Culture Secretary Maria Miller's apology to Parliament.
She was ordered to apologise after a committee found her "attitude" to an inquiry into her expenses had breached the parliamentary code of conduct.
Labour said her 32-second statement, in which she accepted the committee's findings and apologised unreservedly, was "contemptuous" in its brevity.
But Mr Cameron said she had been cleared of the main charge against her.
"I think we should leave it there," he said.
On Thursday, the Commons Standards Committee ruled that Mrs Miller had supplied the Parliamentary Commissioner for Standards, whose role it is to investigate allegations of abuses of the rules on expenses, with "incomplete documentation and fragmentary information".
It added that "officials would press her for information and the information that was provided appears to have been the minimum necessary", and suggested she had tried to argue her case in a "legalistic" way.
The commissioner had found that she over-claimed by £45,000 for expenses towards mortgage interest payments and council tax.
But the committee, which comprises five Conservative MPs, four Labour MPs, one Lib Dem MP and three members of the public, decided that she needed to pay back just £5,800.
She had inadvertently allowed her claims to remain the same when her tracker mortgage fell with interest rates in 2008, Mrs Miller said.
Both the explanation and the figure were accepted by the committee.
Labour's shadow Cabinet Office minister Chi Onwurah said: "We had a contemptuous 32-second response from a cabinet minister followed by a pat on the back from the prime minister.
"It is yet another example of how Cameron only looks after one of his own and always stands up for the wrong people."
The Daily Telegraph reported in December 2012 that Mrs Miller had claimed £90,000 in expenses towards the mortgage on the south London property where her parents lived, while designating her main home as a rented property in her constituency of Basingstoke.
Labour MP John Mann subsequently made a formal complaint to the standards committee.
Following the inquiry, another Labour backbencher, Thomas Docherty, said he had written to the Metropolitan Police asking them to investigate the matter further.
"I am not criticising the committee, but there is a difference between what the committee saw and what the commissioner saw," he said.
The commissioner had had to spend "a long time dragging the information out of Maria Miller", he said.
"If one of my constituents had claimed £6,000 of taxpayers' money for, say, housing benefit and then tried to claim that it was an oversight, that she hadn't realised that she wasn't entitled to this money, the book would be thrown at my constituent, rightly.
"The only way we're going to resolve this, only way we'll get some reassurance for the public, is for the Metropolitan Police to now investigate her.
"I do not understand why Maria Miller still thinks she has done nothing wrong."
The former editor of the Telegraph, Tony Gallagher, told BBC Radio 4's Today programme: "Reading the 113-page report it's very hard to see why she has been cleared.
"The commissioner in fact concluded that she had made a very substantial gain and recommended that she should be paying north of £40,000, and the language throughout the report is utterly condemnatory of the minister.
"The fact that the MPs have then marked their own homework and decided to give her the benefit of the doubt and suggested that she should only be made to repay a mere £5,000 is neither here nor there - the language of the report is shocking in its tone, and I think Maria Miller stands condemned by her own actions."
He also accused the prime minister's official spokesman of threatening the Daily Telegraph with tougher press regulation when it was investigating Mrs Miller's expenses in 2012, shortly after the Leveson inquiry into press standards.
"I got a call from Craig Oliver pointing out that she [Maria Miller] is looking at Leveson," he said.
"These seem to be examples of an attempt to lean on a newspaper and prevent it going about its legitimate business," Mr Gallagher said.
Mr Cameron was asked on a visit to Dawlish, in Devon, whether the allegation against his director of communications was true.
"No," he replied. Mr Oliver, a former BBC executive who began working for No 10 in 2011, said earlier: "It is utterly false for Tony Gallagher to suggest he was threatened over Leveson by me in any way."
Instead, the intervention had been because reporters working on the story were putting "inappropriate" pressure on Mrs Miller's elderly father, Mr Oliver added.
The PM also stood by his culture secretary. "What happened [on Thursday] is that Maria Miller was actually cleared of the original charge made against her.
"It was found she had made mistakes, she accepted that, repaid the money, she apologised unreservedly to the House of Commons, so I think we should leave it there."
BBC political correspondent Carole Walker said that most Conservative MPs attributed the backlash in newspapers against Maria Miller to her support for a new system of press regulation underpinned by a royal charter.
They hope she will ride out the storm and think the PM is right to back her, our correspondent added.
In a short statement to the House of Commons on Thursday, Mrs Miller said she "fully" accepted the committee's conclusions and apologised "unreservedly".
A spokesman for Maria Miller added: "There was a 16-month long inquiry into an allegation made by a Labour MP, at the end of which, the committee and indeed the commissioner, dismissed the allegation."
- The Parliamentary Commissioner for Standards concluded that Mrs Miller had over-claimed by £45,000, but the MP said the true value was £5,800 - and was backed by the standards committee, which is the final arbiter.
- The discrepancy between their conclusions springs from the difference between the size of Mrs Miller's mortgage, which was £525,000 when she entered the Commons in 2005, and the purchase price of the five-bedroom property, which Mrs Miller said was £237,500.
- The commissioner believed she should only have been able to claim expenses for interest payments on the original mortgage of £215,000 - taken out in 1996 - and based her calculations on that.
- The expenses rules in place in 2005, when Mrs Miler was elected, said MPs could not claim for "interest on any additional mortgages, advances or loans secured on the same property".
- In July 2006, the rules were changed and MPs were allowed to claim for "increases to mortgage costs (ie: re-mortgaging) to pay for improvements to a property" - if they obtained permission from Commons officials first.
- There was "no evidence" Mrs Miller had sought such permission, the commissioner said, and even if she had "the rules would not in my view have permitted claims for the interest on a loan which partly or wholly paid for improvements undertaken in the years before she became a Member".
- But the committee disagreed with this conclusion. One of its members, Lib Dem MP Sir Nick Harvey, had served on the committee when the rules were drawn up, and said there had been no consideration of their effect on MPs in Mrs Miller's position.
- The commissioner's reasoning could have perverse consequences, the committee said: Why should an MP who had bought a property on the cheap and did it up before entering politics receive less than an MP from a wealthier background who had simply bought a pricier home to begin with?
- The committee concluded therefore that "a strict interpretation of the rule would not be appropriate" and Mrs Miller had been entitled to claim expenses for interest on the much larger mortgage she had on entering the Commons.