Jimmy Savile abused 63 people connected to Stoke Mandeville Hospital, but the one formal complaint made was ignored, an independent report has found.
It found Savile's reputation as a "sex pest" was an "open secret" among some staff - but allegations probably did not reach managers.
The formal complaint - made in 1977 by an 11-year-old girl's father - should have been reported to police, it added.
A separate report said "elements of the Savile story" could happen again.
The Stoke Mandeville report said the victims, abused from 1968-92, were aged eight to 40.
Sexual abuse by Savile ranged from inappropriate touching to rape - including the rape of children under the age of 12.
Health Secretary Jeremy Hunt said people were "too dazzled or too intimidated to confront the evil predator we now know he was".
The report found:
- Savile had "virtually unrestricted access" to clinical areas and patients during the 1970s and 80s
- several sex abuse claims were made against him from 1972-85, to different staff members, but only one was a "formal complaint"
- that a complaint by a father "should have led to Savile's suspension from the hospital and a formal police report being made"
- there was probably no "hospital-wide intelligence" on Savile
- information known to junior staff and middle managers was "probably filtered out" before reaching senior managers
- over the past 40 years Stoke Mandeville had employed three doctors who had "subsequently been convicted of sex crimes against patients"
Dr Androulla Johnstone, the report's lead investigator, said the victims were "patients, staff, visitors, volunteers and charity fundraisers" - with almost half aged under 16 and 10 under the age of 12.
"Around one third of his attacks were against patients, just over 90% of the victims were female," she said.
"Savile was an opportunistic predator who could also on occasions show a high degree of pre-meditation when planning attacks on his victims."
However, in a statement, the victim of the 1977 formal complaint said she felt "let down and betrayed by the report".
She told the inquiry how Savile sexually assaulted her following an operation at Stoke Mandeville to treat skin cancer, when she was aged 11.
But she said parts of her evidence had been omitted from the report and criticised authors for concluding that senior management would not have known about her father's complaint.
"How they can say that those in authority did not know is inconceivable. I feel that I am being blamed for not taking it further, but I was just a child, suffering from cancer.
"The report is nothing but a whitewash," she added.
Jimmy Savile and Stoke Mandeville
37 spoke to inquiry
19 victims were patients
10 victims were under 12 years old
1 made an official complaint
Savile met Margaret Thatcher, then prime minister, to discuss the funding of Stoke Mandeville's spinal injuries unit in 1980 - and in the same year was appointed to oversee fundraising and rebuilding of the unit.
Lead investigator Dr Johnstone said supervision of Savile at this time was "absent" and - with "statutory functions… in the hands of a celebrity fundraiser" - government and NHS "lost control" of the project.
The Stoke Mandeville report said the hospital trust had tackled Savile's power and influence there "head on" from 1991, and managed to "diminish his authority" - but it took until 1999 to "resolve the situation".
"This says a great deal about the power of the man and the legacy of the historical permissions that had been given to him," it said.
'Weakness for celebrities'
In another report, also newly published, former barrister and NHS executive Kate Lampard reviewed how Savile could have abused victims at 41 NHS hospitals.
"While it might be tempting to dismiss the Savile case as wholly exceptional, a unique result of the perfect storm of circumstances, the evidence we have gathered indicates that there are many elements of the Savile story that could be repeated in future," the Lampard report said.
"There is always a risk of the abuse, including sexual abuse, of people in hospitals."
The report said society had a "weakness for celebrities" and hospitals must be aware of the risks.
A further report highlighted allegations of abuse and rape against Savile's older brother Johnny, who worked at Springfield Hospital as a recreation officer, in south London, in the 1970s.
It revealed allegations by five women against Johnny Savile, who died in 1998, dating back to between 1978 and 1980. The allegations were uncovered by Operation Yewtree officers.
By Nick Triggle, BBC health correspondent
It is easy to say Jimmy Savile's abuse of patients, staff and visitors on NHS premises would not happen in today's health service.
But the report published by Kate Lampard on the lessons learned from the 44 NHS investigations shows this would be dangerously complacent.
While the majority of the crimes took place decades ago, Ms Lampard still warns there are "many elements" that could be repeated.
She goes on to cite a host of weaknesses - from how the NHS deals with celebrities to policies on internet use and social media.
There is, she says, always a risk of abuse - including sexual abuse - in the NHS.
But the difficulty is this: how do you make what is essentially a public space safe when it's so large?
The floor space of the NHS stretches across 26 million sq m (280 million sq ft) - that is enough to cover the City of London nearly 10 times over.
Responding to the reports, Mr Hunt said no system "can ever be totally secure from a manipulative and deceitful predator like Savile".
"We learned last year that there were clear failings in the security, culture and processes of many NHS organisations, allowing terrible abuse to continue unchecked over many years.
"What happened was horrific, caused immeasurable and often permanent damage, and betrayed vulnerable people who trusted us to keep them safe. We let them down."
He told the Commons he would be accepting 13 recommendations made in Ms Lampard's report.
But he would not be accepting one more recommendation that all NHS volunteers should be subject to enhanced Disclosure and Barring Service (DBS) checks, he said.
Mr Hunt said £4 million will be made available from Savile's estate and charities for compensation claims.
Labour's shadow health secretary Andy Burnham said: "The question that will be growing in the minds of people hearing this news today is this: Where is the accountability?
"That is what victims are crying out for and that is what must follow."
'Very different place'
Hattie Llewelyn-Davies, chairwoman of Buckinghamshire Healthcare NHS Trust, apologies to "all of Jimmy Savile's victims".
"I know how difficult it must have been for you to come forward and tell your stories after such a long time."
She said Stoke Mandeville was now a "very different place" but no "sense of complacency" would be allowed.
Liz Dux, of law firm Slater and Gordon, which represents 44 of Savile's victims, said: "It beggars belief that a report which has revealed Savile was widely known as a sex pest at Stoke Mandeville can find no evidence of management responsibility."
She said victims deserved "more accountability".
Simultaneously, the Department for Education has published a string of reports by local authorities into allegations of abuse at a number of children's homes and schools.
In a written statement, Children's Minister Edward Timpson said that although the investigations were complete they had been unable to substantiate any of the allegations.
He said: "None of the investigations have been able to reach firm conclusions about whether the alleged abuse took place or not.
"Although many of them say the informant was credible, the lack of corroborating evidence has prevented them from reaching a definitive conclusion."