Jeremy Corbyn has sought to play down divisions within his top team after one of his closest aides said he would quit and criticised the party's leadership.
Andrew Fisher's exit comes after a failed bid to oust deputy leader Tom Watson, as Labour conference begins.
Mr Corbyn said he got on well with both men and Mr Fisher was "extremely distressed" when he wrote a memo saying the leader's office was "incompetent".
He said he would serve five years if elected PM, adding: "Why wouldn't I?"
On the second day of its conference, Labour is unveiling plans to scrap Ofsted and replace it with a new school inspection system.
Mr Corbyn said the regulator was too "assertive" and its system of oversight needed to be more "supportive" of schools and pupils.
Labour is also promising to axe prescription charges in England if the party wins power, taking it in line with Scotland, Wales and Northern Ireland, where they are already free.
In a wide-ranging interview with the BBC's Andrew Marr, its leader dismissed talk he could stand down as Labour leader in the next year or so as "wishful thinking".
He also defended the party's Brexit policy - to be debated later - amid calls for him to come out unambiguously to remain in the EU rather than sit on the fence.
While most Labour supporters wanted to remain in the EU, he said the party must respect the result of the Brexit referendum and do more to understand why people voted to leave.
If it wins power, Labour would negotiate a new Brexit deal in three months, which would then be put to the people in a referendum within six months, with the option to leave or remain.
Mr Corbyn would not be drawn on which side he would back, saying "let's see" what kind of new deal he was able to negotiate with the EU.
However, he suggested he would ultimately go along with whatever party members decided at a special conference which could be held to settle the issue.
At a fringe event at the party's conference, deputy leader Tom Watson said Labour was a "remain party" and should lead the campaign to remain in the EU in a second referendum.
"By backing a people's vote, by backing remain, I am sure we can deliver the Labour government the people of this country so badly need," he said.
BBC political correspondent Iain Watson says the NEC, Labour's governing body, agreed Brexit proposals on Sunday.
I'm told @UKLabour NEC has now formally backed the #brexit proposals I reported @bbcnews yesterday though one tweak makes explicit that if #remain wins in a referendum a lab govt would push for reform as EU members & a special conf to be called on backing leave or remain #Lab19— iain watson (@iainjwatson) September 22, 2019
Labour conference will be voting on that motion and a Brexit motion on an issue put forward by members on Monday.
Ahead of next week's Supreme Court's ruling on whether the suspension of Parliament is lawful or not, Mr Corbyn said if the judges found against Boris Johnson, MPs must be recalled.
If that happened, he said he would "take immediate action" in Parliament along with other opposition parties to put pressure on the prime minister.
But Conservative chairman James Cleverly said Mr Corbyn could not say whether he would back Brexit even if the party negotiated its own deal.
"Jeremy Corbyn can't even make up his mind on the most important issue facing the country. He would delay Brexit until at least 2020 and even longer if the EU demand it."
'Lack of decency'
Mr Corbyn was dealt a blow on Saturday when it emerged one of his aides, head of policy Andrew Fisher, revealed he will quit his post by the end of the year.
He said he wanted "to spend more time with his young family", but the Sunday Times claims he warned Mr Corbyn would not win the next general election and criticised the leader's office "lack of professionalism, competence and human decency".
Mr Corbyn acknowledged Mr Fisher, who helped write the 2017 manifesto, had expressed concerns about the party's direction and he had spoken to him "at length" about it.
He said Mr Fisher was "extremely distressed" when he made the comments, suggesting it was the sort of disagreement which happened in many workplaces.
"He is a great colleague, he is a great friend. We get along absolutely very well. He has promised whatever happens in the future, we will work together on policy issues."
Amid continuing fallout from the bid to oust Mr Watson, Mr Corbyn also said he was not told beforehand of Friday's move by left-wingers on Labour's ruling body to abolish the role.
The party will now consult on replacing the single role with two deputies - one of whom will be a woman.
Mr Corbyn, who has been at odds with his deputy over Brexit, said he got on "absolutely fine" with him and suggested his intervention had "put the issue to bed".