Junior minister Douglas Ross has resigned after Dominic Cummings' defence of his trip to County Durham during the coronavirus lockdown.
The Scotland Office minister said the senior aide's view of the government guidance was "not shared by the vast majority of people".
No 10 said the prime minister regretted Mr Ross's decision to stand down.
It comes as more than 35 Tory MPs have called on Mr Cummings to resign.
Mr Cummings' decision in March to drive 260 miles from his London home to his parents' farm with his child and ill wife - which he explained on Monday was for childcare purposes - dominated the government's daily coronavirus press briefing.
Asked by a member of the public whether ministers would review penalty fines imposed on families who travel for childcare, Health Secretary Matt Hancock said: "I will have to talk to my Treasury colleagues before I can answer [that] in full and we will look at it."
The BBC's political editor Laura Kuenssberg, citing a government source, said Mr Hancock did "not announce a review" but would pass the concern on to his colleagues.
Rev Martin Poole, a vicar from Brighton, said he asked the question of Mr Hancock because "people feel a bit cheated" and many feel a sense of "unfairness" about the story, adding: "We want to all be treated on a level playing field."
During the No 10 briefing, Mr Hancock said he understood the "anger that some people feel" over Mr Cummings' actions, but added: "My view is that what he did was within the guidelines."
Mr Ross, who remains Conservative MP for Moray, said Mr Cummings' "intentions may have been well meaning" - but that he could not tell constituents who had been unable to visit sick relatives during lockdown that "they were all wrong and one senior adviser to the government was right".
Mr Ross' decision was praised by Labour's shadow Scottish secretary Ian Murray and the SNP's Westminster leader Ian Blackford, who called it the "decent thing" and a "difficult decision" respectively.
At a news conference on Monday afternoon, Mr Cummings said he did not regret his actions and believed he acted reasonably and legally.
Asked why, once in County Durham, he drove his family to the town of Barnard Castle - 15 days after he had displayed coronavirus symptoms himself - Mr Cummings said he had experienced vision problems during his illness and was testing his eyesight to see if he could drive back to London.
The drip drip of Conservative MPs calling on Dominic Cummings to go has continued on Tuesday.
Now surpassing 35, it is around 10% of the parliamentary party.
However, what's notable is that there are those who, even if they're not calling on Mr Cummings to go, have felt it necessary to write long open letters explaining their thinking to constituents.
Public anger, it seems, has not been put to bed by Monday's extraordinary rose garden press conference.
The prime minister's chief aide does, of course, have his backers; people who believe he did what was right in difficult circumstances.
And one government minister suggested to me that the story has been "whipped up" by those who simply do not like Mr Cummings, either politically or as a person.
But this saga is now into its fourth day, in a week when the prime minister wishes to communicate crucial messages about his plans for easing the lockdown.
It is - another minister conceded - a "problem" and "distraction".
And on Tuesday, as yesterday, the question remains, how much political capital is Boris Johnson ready to expend on keeping his chief aide?
Boris Johnson's spokesman reiterated the prime minister's support for Mr Cummings on Tuesday, saying the adviser had "answered questions extensively", while Cabinet Office minister Michael Gove said his account was "exhaustive, detailed and verifiable".
Scottish Conservative leader Jackson Carlaw told the BBC Mr Cummings should resign as the row is "distracting attention" from efforts to combat the coronavirus.
Among the Tory MPs calling for Mr Cummings' resignation is former Attorney General Jeremy Wright, who said combating the coronavirus was "more important than the position of any individual in Downing Street".
He is joined by William Wragg, MP for Hazel Grove, who said it was "humiliating" to see ministers defending Mr Cummings, and Sir Roger Gale, MP for North Thanet, who said the adviser had sent out a "dangerous message".
Former Conservative health secretary Jeremy Hunt told his South West Surrey constituents that Mr Cummings' actions were "a clear breach of the lockdown rules" - but they were "mistakes" and he would not call for his resignation.
Six opposition leaders have said in a letter to the prime minister that removing Mr Cummings from his post "without further delay" is the only way to restore trust in public health advice.
The leaders of the SNP, Liberal Democrats, Plaid Cymru, SDLP, Green Party and Alliance Party said the issue "transcends politics".
"It has united people of every party and political persuasion, who believe strongly that it is now your responsibility as prime minister to return clarity and trust in public health messaging," the letter read.
Meanwhile, the retired chemistry teacher who recognised Mr Cummings in County Durham on 12 April told BBC Radio Newcastle he has some regrets about his involvement.
Robin Lees said he had had a "difficult few days" after his account of the encounter was initially rejected by Downing Street, but that he felt "vindicated" by the subsequent admission.
Latest government figures show the number of people to die with coronavirus in the UK rose by 134 to 37,048 on Tuesday.
There were no Covid-19 related deaths reported in Northern Ireland for the first time since 18 March.
In other developments:
- During the daily briefing, Mr Hancock said there will be "local lockdowns" for "local flare-ups" of coronavirus in the future
- He announced that the government has signed contracts to manufacture two billion items of PPE in the UK
- 59% of British voters believe Mr Cummings broke the rules and should resign, according to snap research carried out by polling company YouGov
- Senior bishops have reported receiving death threats after criticising Mr Cummings' actions
- A drug treatment that appears to shorten recovery time for people with coronavirus is being made available on the NHS
- The Scottish government's new "test and protect" strategy will go live on Thursday, First Minister Nicola Sturgeon has confirmed