Conservative co-chair Oliver Dowden has resigned following two by-election losses for the party to Labour and the Liberal Democrats.
The Lib Dems overturned a huge Tory majority in Tiverton and Honiton, Devon, their third by-election victory over Boris Johnson's party in a year.
And Labour retook the seat of Wakefield, West Yorkshire, which it lost at the 2019 general election.
The prime minister said he would "keep going" and address people's concerns.
In a letter to Mr Johnson, Mr Dowden said Tory supporters were "distressed and disappointed".
He wrote: "We cannot carry on with business as usual. Somebody must take responsibility and I have concluded that, in these circumstances, it would not be right for me to remain in office."
Mr Dowden also called his resignation "a deeply personal decision that I have taken alone" following a "run of very poor results for our party".
The by-election results follow months of criticism of the prime minister over parties in Downing Street during lockdown.
They also come amid soaring inflation and concerns over the cost of living, and after a narrower-than-expected vote of confidence by Tory MPs in Mr Johnson earlier this month.
Speaking in Rwanda, where he is attending a Commonwealth heads of government meeting, Mr Johnson said: "We've got to recognise there is more we've got to do and we certainly will.
"We will keep going, addressing the concerns of people until we get through this patch."
And, in a letter to Mr Dowden, he wrote that he understood his "disappointment" at the by-election losses, but added that the government had been "elected with a historic mandate just over two years ago".
Chancellor Rishi Sunak also expressed regret over Mr Dowden's departure.
"We all take responsibility for the results and I'm determined to continue working to tackle the cost of living," he added.
It is understood there are no plans for the PM to return to the UK and cut short his diplomatic trip - and he has been in touch with senior members of his cabinet.
Some have expressed support, with Deputy PM Dominic Raab saying the government needed to be "relentlessly focused on delivery".
But former Conservative leader Michael Howard told the BBC's World at One Mr Johnson should resign adding "members of the cabinet should very carefully consider their positions".
Leader of the Welsh Conservatives Andrew RT Davies said prime ministers "have to look in the mirror and ask themselves 'can they continue to deliver for their country and for the people who have put them into office?'"
Ben Houchen - Conservative mayor of Tees Valley - warned his colleagues against a "knee jerk reaction" and said that holding a leadership challenge during a cost of living crisis would make the party look "ridiculous".
In Tiverton and Honiton, where former MP Neil Parish quit after he was found watching pornography in the House of Commons, the Lib Dems took 22,537 votes, beating the Conservatives by 6,144.
At the last general election, the Tories had a 24,239 majority in the mostly rural constituency.
In his victory speech, the Lib Dems' new MP Richard Foord said: "Tonight, the people of Tiverton and Honiton have spoken for Britain.
"They've sent a loud and clear message: It's time for Boris Johnson to go, and go now."
Party leader Sir Ed Davey added: "The Liberal Democrats have made political history with this stunning win. It is the biggest by-election victory our country has ever seen."
The Lib Dems also won by-elections in Chesham and Amersham and North Shropshire last year, taking what had previously been safe Conservative seats.
A Conservative source called the result in Tiverton and Honiton "disappointing but not unexpected", telling the BBC the party was confident it could retake the seat at the next general election.
Nothing reeks of panic quite like a resignation letter at 5:35am.
Not just any resignation letter. But the party chairman. Until now, at least, utterly loyal to Boris Johnson.
He manages five paragraphs, not one of which endorses him.
So, does he have confidence in Mr Johnson? He and his team are silent.
What we've seen overnight in miniature is the electoral pincer movement the Conservatives most fear.
Labour's return to seats like Wakefield it could once nonchalantly bank.
The Liberal Democrats planting flags in England's south west once again; a region that not long ago sent a minibus full of such MPs to Westminster.
Conservative MPs from the top down have the jitters this morning; the dawn decision of their former chairman quickening their pulse further.
In Wakefield, where a Labour victory had been largely expected, the party's candidate, Simon Lightwood, won by 4,925 votes.
The previous MP, Imran Ahmad Khan, resigned after being convicted of sexually assaulting a 15-year-old boy.
Visiting Wakefield, Labour leader Sir Keir Starmer said the result there "tells you that the next government is going to be a Labour government".
"And the sooner the better," he added, "because the country voted yesterday, in both by-elections, no confidence in this out-of-touch, out-of-ideas government."
The Conservatives were "absolutely imploding", he said, following Mr Dowden's resignation.
But Mr Raab told BBC Radio 4's Today programme: "My view is that the by-elections, both of them, were the result of the perfect storm of very difficult local scenarios."
The results had followed "situations [involving] the previously sitting Conservative MPs", and come amid "national headwinds" and "the distractions that we've had", he said.
At the 2019 general election, Labour lost Wakefield - one of its so-called "red wall" of safe seats in northern England and the Midlands - for the first time since 1932.