Lord Heseltine has said he would be "very surprised" if Boris Johnson became prime minister after his "preposterous, obscene" remarks during the European Union referendum campaign.
The ex-deputy prime minister, who is campaigning for Remain, said he feared Mr Johnson's "judgement is going".
It marked a further escalation of a war of words between Tories over the EU.
Mr Johnson earlier said it was a "bit too much" for David Cameron to say so-called IS would welcome an Out vote.
And in response to Lord Heseltine's comments a spokesman for Mr Johnson said it was "the arguments that matter".
In other EU developments ahead of the 23 June referendum:
- UKIP leader Nigel Farage raises the prospect of second referendum
- Mr Johnson accuses David Cameron of a "stitch-up" over business backing for the EU
- Shadow chancellor John McDonnell appeals to Labour voters to make a positive case to remain in EU
- Energy minister Andrea Leadsom says leaving the EU will "help keep bills down"
- The European Council president says the only alternative to the EU is "chaos"
- Follow the latest updates on the EU referendum
On Sunday Mr Johnson, the former London mayor, compared the EU's aims in creating a "single authority" in Europe with Napoleon's and Hitler's.
He also suggested on Tuesday that David Cameron was "colluding" with business in the run-up to the 23 June referendum after a leaked letter from the boss of outsourcing firm Serco to the prime minister raised the question of business backing for the UK's EU membership as well as the further privatisation of the prison system.
Speaking to the BBC's political editor Laura Kuenssberg, Lord Heseltine said the "strain" of the referendum campaign was "beginning to tell" on Mr Johnson and he had begun to make "preposterous obscene political remarks".
"He is behaving now irresponsibly, recklessly and I fear that his judgement is going," he said.
Analysis by Laura Kuenssberg, BBC political editor
As Boris Johnson might say: "Cripes!" But that doesn't quite begin to cover it. This criticism will sting because even though Boris Johnson insists it is not the case, many of his Tory colleagues believe his calculation to join the Out campaign was entirely because he wants to be the next leader of his party.
For Lord Heseltine, who of course had similar ambitions but was thwarted in the end, to suggest his decision and subsequent behaviour will in fact kill his chances will really hurt.
Asked if Mr Johnson could lead the Conservative Party one day, Lord Heseltine said: "I'd be very surprised."
He added: "I think that every time he makes one of these extraordinary utterances, people in the Conservative Party will question whether he now has the judgement for that role."
Asked about the historical parallels that Mr Johnson had drawn between the EU and the Nazis, Lord Heseltine - whose challenge helped trigger Margaret Thatcher's departure from Downing Street - said his generation had lived through the war and "knew what Hitler was about".
"When he (Boris Johnson) starts invoking the memories of Hitler, that has crossed the bounds of domestic debate," he said.
"It was about the most manic nationalist aggressive destruction on a scale unprecedented in human history. It was about the persecution of the Jews. A calculated decision to persecute the Jews on a massive scale - that was what he wanted to do. He believed in it.
"The idea that a serious British politician can in any way invoke that memory, I find, frankly, I had better contain my language."
'Arguments not personalities'
Lord Heseltine also accused Mr Johnson of making a "near-racist remark" about US President Barack Obama after he referred in a newspaper article last month to Mr Obama's "part-Kenyan" ancestry and questioned whether this had influenced his attitude to the UK.
In response, a spokesman for Mr Johnson said: "What matters here are the arguments. The British people want to hear debate - they aren't interested in personality politics or personal attacks. Let's get on and discuss the issues."
Lord Heseltine is one of the most prominent pro-European voices in the Conservative Party, having argued for the UK to ditch the pound and adopt the euro.
The Conservative peer rejected claims that both sides were equally guilty of making hyperbolic claims - such as David Cameron's recent suggestion that EU exit could threaten peace in Europe.
It was not scaremongering, he suggested, to tell people what the UK's friends and partners around the world were saying.