A coroner's "high-handedness and aggression" made him unfit to continue in his post, the High Court has ruled.
Two judges upheld a decision by the Lord Chancellor and Lord Chief Justice to remove the Avon coroner, Paul Forrest, from office.
They said the decision was "not marred by any legal flaw" and refused Mr Forrest, 68, permission to seek judicial review of his sacking.
But he will be given time to appeal to the Court of Appeal against the ruling.
Lord Justice Laws, sitting with Mr Justice Burnett, said Mr Forrest's fixed idea about "the scope of his empire" also made him unfit for the post.
Any moves to implement his dismissal were put on hold to give him time to appeal.
Mr Forrest, who also holds office as an immigration judge, has been fighting to retain his post for three years
Problems began for Mr Forrest, who was appointed coroner for Avon in July 1992, when the county was abolished in 1996.
He continued in office, but Bristol City Council took over from Avon County Council as employers of his staff.
Lord Justice Laws said Mr Forrest would say the city council were "merely nominally" the employers.
He became embroiled in a long-running dispute with the city council's head of legal services, Stephen McNamara, over the extent of his control over his office, staff and expenditure.
The judge said it was true that the arrangements for funding coronial services in England and Wales were to some extent "wrapped in history".
A local authority and coroner might have very different views about items of expenditure or directions given to staff, he said.
"The possibility of such differences, however, cannot begin to justify the high-handedness and aggression exhibited by the (coroner) in this case, or the absolute view he takes as to his powers over the coronial staff," he added.
Lord Justice Laws said the city council should, in some instances, "have shown greater sensitivity to the claimant's autonomy as a coroner", but that did not change the outcome of the case.
He ruled: "This is an unhappy case. The claimant, who is not in good health, is without doubt wholly committed to the integrity of his judicial office.
"But he has harboured an 'idee fixe' about the scope of his empire; and as the defendants (the Lord Chancellor and Lord Chief Justice) have concluded, that has led him to behave in a manner that is not compatible with his continuing as coroner."
A major flashpoint was the July 2005 submission of a bid to the Home Office to fund a new forensic mortuary at Flax Bourton, which ultimately failed.
Mr Forrest blamed the city council for this, claiming it "failed to competently pursue a necessary application for planning permission".
Disciplinary proceedings against him began in January 2007 and, in August 2008, an investigating judge concluded the coroner was guilty of "serious misconduct" and recommended his removal from office.
Marina Wheeler, representing the Lord Chancellor and the Lord Chief Justice, said the review body which then looked into the case in August 2009 found that Mr Forrest had shown himself "lacking in good judgment and dogmatic".