Justice Secretary Chris Grayling has admitted that prisons in England and Wales face problems with violence, suicides and staff shortages.
But he maintained there was "not a crisis in our prisons", saying the government was meeting the challenges of a rising prison population.
It comes as the Isis Young Offenders Institution in London is criticised by inspectors in an official report.
High levels of violence were reported at the prison, often involving weapons.
Official figures from the Ministry of Justice (MoJ) show the number of assaults by prisoners in England and Wales rose to 15,033 in 2013-14 from 14,083 in 2012-13.
The latest statistics also reveal a record number of serious assaults - including attacks by prisoners on staff.
But Mr Grayling said prison violence was "at a lower level today than it was five years ago".
Mr Grayling admitted to staff shortages in certain prisons and said there had been "far too many" suicides among inmates, but maintained the government was adapting to such issues.
"We've got challenges of an increased population that was not expected in the last 12 months," he told BBC Radio 4's Today programme.
Mr Grayling added: "We're meeting those challenges, we're recruiting those staff but I'm absolutely clear - there is not a crisis in our prisons.
"There are pressures which we're facing but there's not a crisis."
According the latest figures for England and Wales, there was a 2% increase in the prison population, including those on remand - to 85,509 in the 12 months to the end of June 2014.
The justice secretary said he wanted "to continue to deliver a safe and appropriate prison regime in a world where budgets are much lower than they were".
Mr Grayling said when he became justice secretary in September 2012, he had decided against "an option of privatising a significant number of additional prisons".
Instead he had opted for a proposal by prison governors to identify best practice and bring down costs, keeping the estate in the public sector.
But the Prison Governors Association said staffing levels were too low to make this proposal work.
Is there a crisis in our prisons?
Chris Grayling's interview with Today is his first on the issue of prisons since a recent run of stories highlighting issues of overcrowding, violence and death in prisons, which have led to calls to improve the state of the system.
Earlier this month, a government investigator called for more action to tackle suicide among young inmates.
In July, official prison ratings revealed that more than a fifth of prisons in England and Wales were "of concern".
Chief inspector of prisons Nick Hardwick attacked the state of the system in June, claiming "political and policy failure" was behind dangerous overcrowding in publicly-run jails.
Also that month, it emerged that the government had ordered dozens of already full jails to take in more inmates because the prison population was growing faster than expected.
The Ministry of Justice started recruiting a reserve force of prison officers in June this year.
But the Howard League for Penal Reform reported in July that prison officer numbers in England and Wales had been cut by 30% over three years, from 27,650 officer grade staff in September 2010 to 19,325 in September 2013.
Prison Governors Association president Eoin McLennan Murray said the fundamental problem was "this shortage of staff where we haven't been able to recruit in the numbers that we need in order to run safe, decent regimes".
He told Today that when unions advised the government that prisons should remain in the public sector there was an understanding "we would have certain levels of staffing".
"They have not yet been delivered and that is what the government have to do," he said.
He said that if, for instance, a prisoner became ill and had to be taken out of prison to get treatment it might mean closing an entire wing to provide staff to escort them.
"That means you could be locking up anything from 50-150 men for a period of time, it can be for several hours," he added.
"They can lose their associations during the evening, they can lose their association during the weekends. This is happening around the country."
Mr Grayling's comments come as an unannounced inspection of the Isis Young Offenders Institution in Thamesmead, south-east London, reported high levels of fights and assaults.
Of these, it stated, "more than we usually see involved weapons".
"Nearly a third of prisoners felt unsafe", it added, with many incidents in the prison - which houses 18 to 30-year-olds - involving groups attacking single inmates.
One former inmate at a young offenders institute, Justin Rollins, said the environment was brutalising.
"I realised straight away that if you don't fight... if you don't act out with some sort of extreme violence, then you're going to get basically terrorised," he explained.
"It's not a nice place for a kid to be when you're away from your family for the first time."
The report into the Isis Young Offenders Institution also found "obvious tensions" between staff and prisoners due to the effect of staff shortages, with inmates allowed less time outside their cells.
David Murray, a recently retired prison officer with 30 years' experience, said he had seen similar issues elsewhere in the past two years.
"More staff are being assaulted, vulnerable prisoners are in danger of being badly hurt or killed.
"It's frightening to go into an establishment where young people who come from a violent background are not caring what they do to people."
Another prison officer, who works at Feltham Young Offenders Institute, said: "Many of the attacks are out of fear - fear that unless they act tough they will be started on themselves and now we have fewer prison officers to deal with it too," he told BBC News.
Fred Lunn, who was jailed for armed robbery from the age of 18 to 37, said makeshift weapons in prisons can have a "devastating" effect.
"I've seen people boil up cooking oil, I've seen people get that in their face."
The Prison Governors Association says some prison officers are afraid to intervene in case they are accused of assault.
In June, the chief inspector of prisons warned that, because of staff shortages, men were locked up together for 23 hours a day, causing "huge tension".
Nick Hardwick said extra resources were needed or the prison population had to be reduced.
At the time, Mr Grayling told the BBC he disagreed.
He said there were 1,000 spare prison places and promised 2,000 new places by April.
According to MoJ figures, the proportion of prisoners held in crowded conditions in 2013/2014 fell to 22.9% of the prison population - down from 23.3% the previous year.
It said this was the lowest level since 2001/02.