A former Broadmoor Hospital manager says he has been made a "scapegoat" in a report which highlights his links to Jimmy Savile and includes claims of "inappropriate" relations with staff.
Alan Franey, of Welwyn Garden City, Hertfordshire, ran the high security psychiatric hospital from 1989 to 1997.
Mr Franey says the report into Savile's involvement with the Berkshire hospital "taints" him by association.
The Department of Health said the report was "appropriate and robust".
The report says some staff claimed that Mr Franey, who is the Conservative deputy leader of Welwyn Hatfield Borough Council, "put it about a bit" and used his accommodation for "inappropriate activities".
Mr Franey said the claims were "nonsense" and that he was being treated like a "scapegoat".
"(The claims) makes a mockery of the report," he told the BBC.
Newly-published reports have said Savile, who died in 2011, sexually assaulted victims aged five to 75 in NHS hospitals over decades.
The Broadmoor report details 11 allegations of sexual abuse, six of them involving patients.
Mr Franey said the allegations against Savile were "unforgivable" but had he known about them, he would have had "Savile removed".
The report said Mr Franey had denied the allegations against him and that the "various stories of 'about 50' affairs were attempts by those opposed to change to discredit him".
It continued: "It is not necessary for us to form a view on whether there is any substance to these stories or not and there is only second-hand evidence to suggest that there may be.
"What is important, however, is that many staff, at all levels of the organisation, thought that there was substance to them and regarded this as damaging to Franey's stature and credibility."
The report said there was a "strong probability" that the perception among staff "impaired the general manager's ability to lead the organisation".
Mr Franey had been recommended for the post at Broadmoor by Savile, who he met at Leeds General Infirmary and ran charity races with, the report said.
But Mr Franey stressed he went through the proper recruitment procedure to get the job.
A former director of the Buckinghamshire Association for Mental Health, Mr Franey said he hoped the claims would not affect his role as a councillor.
A Department of Health spokeswoman said: "An independent assurance process, overseen by Kate Lampard, concluded that the investigations were conducted in an appropriate and robust fashion."