Boris Johnson has said a "psychological transformation" in his character is "not going to happen" after by-election defeats led to calls for change.
The PM was responding to Tory party chairman Oliver Dowden saying it could not be "business as usual" as he quit.
Mr Johnson told BBC Radio 4's Today programme, he "humbly and sincerely" accepts criticism.
But he said he also had to distinguish between "criticism that really matters and criticism that doesn't".
The by-election defeats in Wakefield and Tiverton and Honiton on Friday came after the prime minister faced months of criticism over parties in Downing Street during lockdown, alongside soaring inflation and a narrower-than-expected win in a confidence vote from his own MPs.
Speaking from the Commonwealth heads of government meeting in Rwanda, Mr Johnson repeatedly said that policy was more important than allegations about his conduct.
Mr Johnson said voters were "fed up with hearing conversation about me" and wanted to focus instead on the cost of living, the economy and "standing up to violence and aggression" in Ukraine.
The PM was challenged by presenter Mishal Husain that a lot of the criticism had been about him personally and had come from people who had worked with him.
They included his top policy aide Munira Murza, who criticised Mr Johnson's "scurrilous allegation" about Jimmy Savile and Sir Keir Starmer; former minister Jesse Norman, who said the PM "presided over a culture of casual law-breaking"; and ethics adviser Lord Geidt.
But Mr Johnson said: "As a leader, you have to try to distinguish between the criticism that really matters and the criticism that doesn't."
Asked if there was any matter of principle he would consider resigning over, he said if he had to abandon Ukraine because it became too difficult or the costs were too great, he would quit.
He said that "of course" he regarded morality as a part of leadership.
But Mr Johnson was questioned about how he had not resigned over misleading the House of Commons, breaking the law for the Covid fine, or losing the support of 41% of his MPs in the confidence vote.
"Let's look at this in a more cheery way, if that's possible" he said. "Actually, what's happened is that I've got a renewed mandate from my colleagues, and I'm going to continue to deliver."
The prime minister did not engage with a question about the UK's top civil servant Simon Case having an informal conversation about job opportunities for his wife, Carrie.
"The worst thing I could do is get into conversations about my family," he said.
This was an interview in which the prime minister repeatedly attempted to detach his character and conduct, from policy.
"You have to try to distinguish between the criticism that really matters and the criticism that doesn't," he said.
There was an acknowledgement his character isn't going to change.
And he sought to turn criticisms of it into policy disagreements - citing, for instance, the one Conservative MP who has advocated the UK re-joining the EU's single market.
His problem is many of his Conservative critics think the judgement of big chunks of the electorate, including among Tory voters, is set when it comes to his character.
And there is restlessness, too, among some backbenchers on policy as well.
He is repeatedly struggling to change the record from one about his own future, and how much of one he has in Downing Street.
Speaking to reporters in the Rwandan capital of Kigali later on Saturday, Mr Johnson was asked if he considered the question of his leadership settled and said: "Yes."
He refused to comment on a report in the Times that he planned to build a £150,000 treehouse for his son Wilf in the grounds of Chequers, the prime minister's country residence, with possible funding from a Tory donor.
"I'm not going to comment on non-existent objects or non-existent jobs to do with my family," he said.
Asked in his BBC interview about policies to ease the rising cost of living, Mr Johnson declined to offer a commitment to cut fuel duty further or remove VAT from energy bills.
"There may be more things we can do," he said, but he added that ministers are already "doing a huge amount with the fiscal firepower we have" and said the public understands the financial situation the government is in.
Tory backbencher Bob Neill told the BBC the prime minister's response to the by-election defeats was wrong.
"This is a genuine sense of growing frustration and growing alarm that the party is being taken down a track that will lead to it inevitably losing the next election," he said.
Criticism of Mr Johnson was not just about "personality", but whether his "character and adherence to the rules" were consistent with standards that Conservative voters in particular expect, Mr Neill said.
He said there was also "evidence of drift and lack of grip on economic issues", while "the highest rate of taxation for 40 years" was not an economic policy Conservatives would recognise.
The prime minister's interview comes as two Tory MPs critical of his record say they may stand for election to the committee that runs the party's leadership contests, with one saying he would be in favour of changing the rules to allow a second confidence vote.
Foreign Secretary Liz Truss, who has joined the PM in Rwanda for the summit of Commonwealth leaders, said people often want to "send a message" to government in by-elections and the defeats were not necessarily a predictor of general election failure.
"It hasn't been the predictor in the past and I don't believe it will be the predictor of the next general election," she said.
Ahead of a meeting of G7 and Nato leaders, she also urged governments to continue to support Ukraine's "fight for freedom", saying any concessions to Russia would be "appalling".
At the summit in Kigali, Mr Johnson also defended his government's policy of trying to deport asylum seekers to the east African nation.
Asked by the BBC by how much Channel crossings in small boats needed to be reduced for the policy to be considered a success, he said: "I'm not going to give you a figure."
He said it was a "humane policy" which was about "breaking the business model of those who criminally abuse and cheat people crossing the Channel in unseaworthy vessels."
- STRUGGLING WITH SLEEP: Is your diet keeping you up at night?
- ELECTRIC OR MANUAL?: Could a more expensive toothbrush clean your teeth better than a cheaper electric one?