Two privately-run prisons are among three the government has expressed "serious concern" over, Ministry of Justice (MoJ) ratings have revealed.
HMP Oakwood, run by G4S, HMP Thameside, run by Serco, and the Prison Service's HMP Winchester in Hampshire, were given the lowest performance rating of one.
Of the 134 prisons in England and Wales, 14 are privately run, including Oakwood and Thameside.
G4S and Serco said issues were being addressed.
The MoJ also said action was being taken but critics called on the government to "halt the privatisation of justice".
The ratings come after private security companies G4S and Serco came under fire for overcharging the government by "tens of millions of pounds" for providing electronic tags for criminals.
It triggered a government-wide review of all contracts held by the two firms.
In the annual assessment a further 12 prisons were said to be "of concern", while the remainder were either rated as having an "exceptional performance" or "meeting the majority of targets".
Separately, every probation trust in England and Wales was rated "good" or "exceptional" by the MoJ.
Oakwood, in the West Midlands, opened in April 2012 and is one of the largest jails in the country, with a capacity for 1,600 male inmates. On its website, the prison says that it aspires to be regarded as "the leading prison in the world" within five years.
A spokeswoman for G4S said the company was working with the MoJ to address issues at the facility, adding: "I am completely confident that Oakwood will go from strength to strength."
Thameside, in London, opened in March 2012 and can hold 900 male prisoners.
Wyn Jones, Serco's director of custodial services, said it was early days at a brand new prison and they were working with the MoJ to "achieve excellent standards".
"It takes times to develop and embed innovative systems, new working practices and initiatives," he said.
Winchester Prison, built in 1846, was criticised by inspectors earlier this year after two elderly disabled men were locked up together for almost 24 hours a day, without easy access to showers.
Inspectors said standards at the jail had "deteriorated sharply".
Responding to the ratings, Frances Crook, chief executive of the Howard League for Penal Reform, said: "There could not be a more damning indictment of the government's fanatical obsession with justice privatisation than its own performance figures.
"Last autumn, the justice secretary hailed G4S Oakwood as an example of what the private sector could achieve in prisons. We agree. The prison, ranked joint-bottom in the country, is wasting millions and creating ever more victims of crime."
An MoJ spokesman said HMPs Thameside and Oakwood were "still in the early stages" and that "experience demonstrates that it takes up to two years for a new prison to develop".
On Winchester, the spokesman said a performance improvement plan was in place and "significant progress" had been made.
The positive ratings for probation services prompted calls for Justice Secretary Chris Grayling to reconsider his promised "rehabilitation revolution", under which private security firms would be paid to supervise low to medium-level offenders across England and Wales.
Juliet Lyon, director of the Prison Reform Trust, said the ratings provided evidence of how well the public probation service was performing, urging the government to reflect on its "ill-conceived plans".
But Mr Grayling said new reoffending figures, which showed more than 400,000 crimes were committed in the year up to September 2011 by people who had broken the law before, made a "compelling case for our important reforms".
Other figures for the year up to September 2011 showed a 1.2% rise - to 58.5% - in the number of criminals sentenced to less than 12 months in prison who reoffended within a year of their release.
Mr Grayling said a pilot project run in Peterborough had led to "real improvements".
This scheme, which the government plans to roll out across England and Wales by 2015, offered private firms "payment by results" for rehabilitating offenders and cut the reoffending rate from 84 convictions per 100 prisoners released to 78 per 100, the MoJ said.