Four men have been held for more than two years at Brook House immigration removal centre as the average length of detentions has grown, a report found.
Prison inspectors discovered the average period of stay at the facility near Gatwick Airport had risen from 28 to 48 days.
Twenty-three people were held for more than a year.
The Home Office said it did not want to detain people for longer than necessary but some had prolonged the process.
Brook House holds almost 400 adult male asylum seekers, illegal immigrants and foreign national offenders.
Despite a push by the Home Office last year to reduce the number of people being detained, inspectors could not identify why periods of detention had lengthened.
The report added that some detainees at the G4S-run centre had been held for excessive periods due to "unreasonable delays in immigration decision making".
Peter Clarke, chief inspector of prisons, said the lack of analysis made it "hard to see how detention periods could be systematically reduced and the inevitably negative outcomes for detainees mitigated".
A 2016 Home Office review said fewer people should be detained in immigration centres, while pointing out that each place cost £34,000 a year.
In response to the report a spokesman said: "Detention is an important tool that helps us remove those with no right to be in the country and it is vital that this is carried out with dignity and respect."
But he said some of those being held had attempted to frustrate the removal process by failing to provide accurate and timely information about their identity.
He said of the people leaving detention in 2016, 64% left within 29 days and 93% in less than four months.
The centre was assessed as "reasonably good" on safety, respect, activities, and preparation for removal or release, but inspectors were concerned that the residential units "very closely resembled" a prison.
Director of the removal centre, Ben Saunders, said inspectors had "rightly" identified the purpose-built design was stark in some places, and G4S would work with the Home Office, which owns the buildings, to soften the environment.