A surprise inspection of an immigration removal centre found there were still "significant concerns" over its treatment of detainees.
Brook House, at Gatwick Airport, was branded as "fundamentally unsafe" in March 2010.
In a recent inspection, Nick Hardwick, Chief Inspector of Prisons, said there was "excessive and often illegitimate" use of the separation unit.
However, he said the centre was safer and more settled than previously.
G4S, which operates Brook House, said staff had worked hard to improve the safety, security and well-being of detainees
The complex, which opened in March 2009, is designed to hold more than 400 men, many of whom are ex-prisoners.
It has a similar style of security to a Category B prison, the second highest level of imprisonment.
Mr Hardwick said: "Managers and staff should be commended for bringing a level of stability to the centre, which is now a largely safe environment run by properly supported staff.
"However, Brook House continues to face a number of challenges and much more needs to be done, particularly in relation to separation, activities and preparation for release and removal.
"The centre had a tough start, but is now past that phase in its development. We expect it to have built on the firmer footing that it has now achieved by the time we return."
He added that inspectors were pleased the number of assaults had reduced, self-harm procedures had improved and detainees had good access to health care.
However, there were concerns that professional interpreting was used sparingly, education was minimal and the practice of taking reserve detainees to fill spaces on escorted flights, and then returning them, continued.
The inspection was carried out in September 2011.
Jerry Petherick, from G4S Care and Justice Services, which operates Brook House, said: "We take the welfare of those in our care extremely seriously.
"We have worked hard to improve the safety, security and well-being of detainees at Brook House. We are pleased this report has recognised the 'substantial improvements' which have taken place since the last inspection and that we have been commended for the level of stability which has been brought to the centre under our management.
"We recognise that challenges remain and we will be reviewing all the recommendations with a view to address outstanding issues."