Boris Johnson has ruled out an inquiry into the conduct of his top adviser at the height of lockdown, insisting it was time to "move on" from the row.
Dominic Cummings is accused of breaking lockdown rules by travelling from London to County Durham.
The prime minister rejected claims he had damaged his own authority and the government's coronavirus message by not sacking Mr Cummings.
He said the public had had enough of the "political ding-dong" over it.
Around 40 Tory MPs have called for Mr Cummings to resign or be fired after his 260-mile journey came to light.
And Cabinet Office Minister Penny Mordaunt has reportedly said there are "inconsistencies" in Mr Cummings' account of his actions during lockdown and there was "no doubt" he took risks.
In an email to her Portsmouth North constituents, seen by The Guardian, she said: "Other families have been faced with the same situation Mr Cummings and chosen to stay put."
She did not call for Mr Cummings to be sacked - but apologised for the way the past few days have "undermined key public health messages".
Shadow home secretary Nick Thomas-Symonds said the prime minister had given "no credible explanation" why evidence had not been passed on to the cabinet secretary to investigate.
The Labour MP added that the prime minister's handling of the story had "undermined the public health message he is trying to put forward".
Trip 'not necessary'
Sajid Javid, who resigned as chancellor in February after Mr Johnson ordered him to fire his team of aides, also called on Mr Cummings to apologise.
In a letter to constituents first reported by the Bromsgrove Standard, he said he did not believe the aide's trip to County Durham was "necessary or justified".
On Tuesday, junior minister Douglas Ross resigned in protest at Mr Cummings' defence of his behaviour, saying that his interpretation of the government guidance was "not shared by the vast majority of people".
At an appearance before the Commons Liaison Committee, Mr Johnson said he "did not propose to add" to his previous statements on Mr Cummings - or what he described as the "autobiography" the aide delivered on Monday.
The prime minister used the phrase "move on" five times in 20 minutes, as he faced hostile questions from select committee chairs.
Asked whether the Cabinet Secretary Sir Mark Sedwill, the UK's top civil servant, should investigate Mr Cummings' actions, he said: "Quite frankly, I am not certain that right now an inquiry into that matter is a very good use of official time.
"We are working flat out on coronavirus."
He said the public wanted the government to "focus on them and their needs, rather than on a political ding-dong about what one adviser may or may not have done".
Asked whether the government's "moral authority" had been undermined by Mr Cummings' actions and his own defence of them, Mr Johnson said: "I, of course, am deeply sorry for all the hurt and pain and anxiety that people have been going through throughout this period - this country has been going through a, frankly, most difficult time.
"We are asking people to do quite exceptionally tough things, separating them from their families."
While Mr Johnson was taking part in a video conference with the panel of senior MPs, Conservative MP Giles Watling tweeted: "I've been listening to the PM in the Liaison Committee. I applaud him for sticking by his man, but I'm afraid Mr Cummings should stand down.
"His continued presence at the heart of government at this time is an unwanted distraction."
Wednesday's session was the first time the prime minister has faced questions from MPs since the allegations against Mr Cummings emerged at the end of last week.
He used his appearance at the committee to announce that NHS England's test and trace system would be up and running from Thursday.
He was also quizzed about schools, care homes and the economy.
Mr Cummings' decision in March to drive from his London home to his parents' farm in County Durham with his wife - who had coronavirus symptoms - and his son has dominated the headlines since the story broke on Friday night.
The PM's chief adviser gave a news conference on Monday, explaining that he decided to make the trip because he felt it would be better to self-isolate in a place where he had options for childcare if required.
He has received the continued support of the prime minister, who said that his aide had acted legally and with integrity.
On Wednesdays Mr Johnson would ordinarily face Labour leader Sir Keir Starmer at Prime Minister's Questions, but MPs are currently on recess.
The Liaison Committee - a panel of 37 MPs who chair various select committees - is the only Commons committee that can question the prime minister.
This was Mr Johnson's first appearance before the committee since he became PM last July.
Content available only in the UK