HMP Birmingham, the first in the UK to be privatised, faces "significant strategic challenges", a report has found.
Chief Inspector of Prisons Nick Hardwick said HMP Birmingham was "a cleaner, safer and more decent place" since it was privatised last October.
But he warned the prison, operated by G4S, needed to improve its training and drugs programmes.
G4S said it welcomed the report and was considering its recommendations.
HMP Birmingham became the first prison to be taken over by a private company after previously being in the public sector in October 2011.
Mr Hardwick said: "Birmingham prison has recently made some relatively simple but nonetheless important improvements but the prison also has a number of significant strategic challenges it needs to resolve.
"It is a cleaner, safer and more decent place. However, first night and vulnerable prisoner arrangements are significant exceptions to that overall picture."
Inspectors said there was "insufficient support" for those wishing to reduce their dependency on drugs, while the single alcohol worker was "overwhelmed".
The report said there had been 54 reported incidents of packages being thrown over the prison wall in the previous three months.
The regime for vulnerable prisoners was described as "unsatisfactory", with many of them facing harassment from other prisoners and with poor access to services.
Overcrowding remained a problem with too many prisoners spending too long in small, shared cells, Mr Hardwick said.
The report found not enough prisoners were learning new skills, or taking part in other activities, while training opportunities were often missed.
Despite the outlined concerns, inspectors praised improvements in safety, cleanliness and relations between prisoners and guards.
There was also praise for the treatment of older prisoners, who have access to a day care centre.
In a statement the company said: "We have made a start at implementing a new regime aimed at improving the rehabilitation of offenders.
"We will look at the recommendations within the report carefully and will continue to further improve the work, training and education available for prisoners."