A "significant number" of England's further education colleges will struggle to remain viable in the next three years, a report warns.
Obtained under freedom of information laws, the report for the former college funding body the Learning and Skills Council, says 50 colleges could close as the sector "down-sizes".
It urges many more to find efficiency savings by merging or federating.
It calls for urgent action to address the financial health of the sector.
The KPMG report for the LSC looks at how colleges could best find value for money savings in the tougher financial climate.
Colleges have been very successful in the past five years with the majority being rated good or outstanding, it says.
But it warns of a small number with poor quality and failed financial health, and a larger number of colleges which are "at risk" of failing in either or both categories.
It quotes figures from the Association of Colleges which, for the second year in a row, says colleges collectively will run a deficit overall. In 2008-9 this was estimated to be £34m.
It argues that in the light of growing financial pressures, every college will need to "deliver greater value for money or risk failing".
"The financial health of the FE sector is, in general, deteriorating rapidly and requires urgent action," it says.
The claim comes against a backdrop of £200m in adult education budgets and a £230m write-off over a mismanaged building development scheme.
It urges the government to drive the changes to the college sector "before serendipity and individual college ambition takes the lead".
But it also calls on colleges to push things forward, adding: "Quality provision in FE must be maintained as the sector down-sizes the number of general further education college providers, possibly by up to 50 colleges."
"For many failing, struggling and vulnerable colleges, merger may either take too long... or represent the last resort," it adds.
It says that in the past, struggling colleges could rely on public sector support, but warns that is unlikely to continue in the present financial climate.
It argues that there are economies of scale linked to mergers, and that in general larger colleges are not only more efficient but have better Ofsted ratings.
Those with annual turnover of less than £35m are particularly vulnerable and should consider merging, the report argues.
Geoff Russell, chief executive of the Skills Funding Agency, which replaced the LSC in April this year, and has just published the report, said the current financial climate meant that all colleges were under pressure to make better provision for less.
"This report will hopefully trigger further debate about how we can all achieve greater efficiencies without affecting the quality of delivery," he said.
"The Skills Funding Agency wants to contribute to sector thinking, not to impose solutions."
He added that it would continue to work with the Department for Business Innovation and Skills in this regard.
Martin Doel, chief executive of the Association of Colleges, said the report was overly bleak, but acknowledged that college income would decline for the next few years.
There may well be fewer or smaller colleges moving forward, he said.
But he added: "Colleges are right to plan for harder times ahead, but they are also quick to seize opportunities.
"Apprenticeships, part-time higher education, courses for 14-year-olds and privately funded adult education are all possible areas of future income growth, in addition to the core areas of 16 to 18 education."