Wormwood Scrubs prison is rat-infested and overcrowded, with some prisoners too frightened to leave their cells, inspectors have said.
A report by HM Inspectorate of Prisons found safety at the prison had deteriorated since a previous probe raised serious concerns.
The jail had "levels of Dickensian squalor", the Prison Reform Trust said.
The National Offender Management Service (NOMS) said the prison had "moved forward" since the inspection.
HM Inspectorate of Prisons' report was carried out between 30 November and 4 December 2015. It was an announced inspection - which is unusual, the inspector said.
Two inmates at the west London jail who were deemed to be at risk of suicide or self-harm were found to be in cells in which jagged glass remained in a broken window.
Some prisoners used a torn sheet as a makeshift toilet screen, while others stuffed paper in to broken windows to "keep out the weather", the report found.
Inspectors said they saw rats "every day and night"of their visit, and one in five prisoners said they felt unsafe.
Chief inspector of prisons, Peter Clarke, said: "Wormwood Scrubs is a prison that continues to fall short of expected standards, and at the time of our inspection there was little cause for optimism."
- About 1,300 prisoners are held at the jail
- Famous ex-inmates include Babyshambles frontman Pete Doherty, incarcerated in 2008 for drug and driving offences
- Rolling Stones stars Keith Richards and Mick Jagger also spent a night there in 1967 on drugs charges
- The prison, dating to 1891, was listed as Grade II in 2009
- Its gatehouse was made famous as the entrance to HMP Slade in the HMP Slade in the 1973 BBC TV comedy series Porridge
Michael Spurr, CEO of NOMS, said the "old, crowded" jail was "challenging to run" but changes were being made.
"Since the inspection the prison is cleaner; more prisoners are going to activities; violence is being tackled; and better support is being provided to vulnerable prisoners", he said.
Juliet Lyon, director of the Prison Reform Trust said: "This shattering report on London's best-known Victorian jail reveals levels of Dickensian squalor which ought to have been consigned to the history books."
The government previously announced plans to close Victorian jails and build nine new prisons, but only HMP Holloway has been confirmed as facing closure.
A Ministry of Justice spokeswoman said it was "investing £1.3bn to transform the prison estate".