Nearly two-thirds of prisons in England and Wales are overcrowded, according to the Prison Reform Trust.
It said there were 7,294 more people in the system than it was originally designed and built to hold.
HMP Kennet in Liverpool is the most overcrowded prison in England and Wales, followed by Shrewsbury and Swansea, according to the trust.
A Prison Service spokesman denied overcrowding and said some prisons have been changed to house more inmates.
He also said that the service was aiming to reduce crowding and the cost of the prison estate.
"All of our prisons provide acceptable levels of accommodation for prisoners, although some prisons hold more people than they were originally designed for," he added.
Although the growth in the prison population has slowed down in recent months, there were 77 out of 131 establishments operating over the Prison Service's "certified normal accommodation" (CNA) as of 27 July, the trust said.
CNA is "the good, decent standard of accommodation that the service aspires to provide all prisoners", a trust spokesman added.
HMP Kennet was designed to hold 175 male prisoners, but holds 337.
The prison opened in 2006 after the National Offender Management Service bought Ashworth East Hospital from Merseyside Care Trust to help tackle overcrowding in prisons in England and Wales.
In second place was Shrewsbury - built to hold 170 men, but holding 326; and third was Swansea which was built for 240 but holds 436.
A trust spokesman said: "For people in prison themselves, overcrowding has a tangible impact."
He said that figures for 2010-11 showed that nearly a quarter of people in prison were being held in overcrowded accommodation, either doubling up in cells designed for one occupant or being held three to a cell in spaces designed for two people.
Private prisons have held a higher percentage of their prisoners in overcrowded accommodation than public sector ones every year for the 13 years to 2010-11, it said.
A spokeswoman from the Ministry of Justice said no prison in England and Wales was working above its operational capacity.
"When a prison is first built it is given a CNA, but over time changes may be made to the prison allowing it to hold more inmates - this is its operational capacity."
According to figures from the Ministry of Justice the operational capacity of Kennett was 342, Shrewsbury 340 and Swansea 435.
The spokeswoman said Swansea may have one too many inmates because a prisoner was in hospital.
A former prisoner of Walton Prison, now called HMP Liverpool, in 2003, who wished to remain anonymous, said he spent three years in an overcrowded prison where he was forced to spend 23 hours in his cell.
"It was like being put in a cage made for one animal and then getting two or three other people in," he told BBC News.
"There was a lot of stress, anxiety and violence. It was one big stress pit is the only way I can describe it."
Liverpool Prison has one of the highest prison populations in England and Wales. The prison's certified normal accommodation is 1,186, it's operational capacity is 1,462. It holds 1,257 prisoners.
Lorna Brookes, who works with prisoners' families in the Merseyside area, said overcrowding at Kennett means inmates do "not engage properly" with the prison regime due to added stress.
"They might lose their privileges, which could mean they can't see their family, which in turn affects their spouses and children," she said.
The prison population currently stands at 86,841, as of Friday, 24 August.
The same time last year, it was 86,821, and five years ago it was 80,534.