Treatment of some people with learning disabilities in hospitals and care homes is "intolerable", a review of the sector in England has found.
Sir Stephen Bubb had recommended moving people to community care and a charter of rights, after the abuse scandal at Winterbourne View care home in 2011.
In his final report, he criticised lack of progress and said a commissioner should be appointed to drive reform.
The government is understood to be looking at all the recommendations.
In 2011, the BBC's Panorama uncovered serious patient abuse and neglect at the Winterbourne View private hospital, near Bristol.
Sir Stephen - who is chief executive of the Association of Chief Executives of Voluntary Organisations - was then asked by NHS England to examine how to address "serious shortcomings" in the support for those with learning disabilities.
In November 2014, Sir Stephen published his report - Winterbourne View - Time for Change - that said many people were being kept in hospitals far from home for far too long.
He made 10 recommendations, including closing large "inappropriate in-patient facilities" in favour of care services for people in their own community and the introduction of a legal charter of rights for them and their families.
But he later reported an "absence of any tangible progress".
'Advocate for change'
Speaking on BBC Radio 4's Today programme, as he published his final report on Monday, Sir Stephen said: "I have been really shocked by what I have heard about seclusion, around over-medication, around the use of physical restraint.
"It's an intolerable way to treat people with learning disabilities.
"The right place for people is with their families and in the community, supported properly."
But Sir Stephen said that what needed to be addressed was "not just around closing institutions", it was also about how society treated people in the education and health systems.
"That's why my major recommendation today is for the establishment of a commissioner for learning disabilities - someone who will drive and act as an advocate for change and for making more progress," he said.
The latest report, titled Time For Change - The Challenge Ahead, says some 3,500 vulnerable disabled people are still in institutions - 30%, or 900, more than had been thought.
It suggests that supporting people in their own community will require 10,000 extra staff, who will need to be trained to an established standard.
With at least 1,300 people expected to move out of hospital care by 2019, the report also says there is a "critical need" to develop housing for vulnerable people who, Sir Stephen says, should be exempt from proposed housing benefit caps.
'Too little progress'
The Department of Health is understood to be looking at all of Sir Stephen's recommendations, including the establishment of a commissioner.
Health minister Alistair Burt said: "It was only recently that NHS England announced a major programme to move people with learning disabilities out of hospital and into their communities...
"This, combined with the increase in specialist staff including nurses, will transform care.
"We are not complacent and will work with the NHS, local government and others to make sure their plan is delivered."
Luciana Berger, Labour's shadow minister for mental health, said there had been "too little progress" since the Winterbourne scandal.
"Tory ministers must avoid repeating their mistakes, accept these proposals and deliver the transformation in our learning disability services that is urgently required," she said.
Timeline: Winterbourne View scandal
- May 2011: BBC Panorama exposes Winterbourne View scandal. The then-coalition government promises to prepare to get thousands of people out of hospitals and into community care by June 2014. The deadline is not met
- July 2014: Sir Stephen Bubb is asked to review care facilities for learning disabled people
- November 2014: Sir Stephen's review is published and recommends the closure of in-hospital care units
- February 2015: The National Audit Office publishes a report saying ministers have underestimated the scale and complexity of the problem. It says ministers no longer have the power to compel health and social services to provide and fund the new bespoke care, because power has been devolved down to the local level
- February 2015: NHS England boss Simon Stevens faces a committee of MPs and commits to create a closure programme for large NHS mental health hospitals, and a transition plan for people with learning disabilities, within six months
- July 2015: Six months on, Sir Stephen says there is an "absence of any tangible progress" and pace of change is slow
- October 2015: NHS England publishes its closure programme in line with the recommendations made by Sir Stephen
- February 2016: Sir Stephen says the government's "failure" to create a charter of rights for people with learning disabilities means a new commissioner is needed to "protect and promote" them.