A new generation of Oxford and Cambridge colleges should be opened to create more places for disadvantaged youngsters, says a report into widening access to university.
The Higher Education Policy Institute (Hepi) has published ideas from MPs, think tanks and academics to make university more inclusive.
The report wants more Oxbridge places, as other universities have expanded.
Oxford and Cambridge both said they had "no plans" to create new colleges.
The report, Reaching the Parts of Society Universities Have Missed, makes recommendations for tackling unequal access to higher education.
'Entirely new colleges'
"People from rich households are more likely to reach the most prestigious institutions, white working-class boys rarely make it to higher education and there is a big black attainment gap," said Hepi director Nick Hillman.
Entry to Oxford and Cambridge is described as "hyper-selective" and the report suggests a practical response would be to create more places, in the way that many other universities have expanded.
"If existing colleges are reluctant to increase their undergraduate entry, then it is time to consider founding a number of entirely new Oxbridge colleges to boost the number of students from under-represented groups at our oldest, richest and most prestigious universities," said Mr Hillman.
Figures from the Higher Education Statistics Agency (Hesa) show many leading universities have expanded rapidly in recent years in their undergraduate intake, some by more than 50%.
But Cambridge has seen only a much more modest rise in first-year numbers - up by 4%.
According to the figures from the higher education data body, Oxford has reduced its first-year intake, down by 21% over the past five years.
Oxford challenges the figures used by Hesa, saying the university's own figures suggest full-time undergraduate numbers have remained broadly similar, with a 2% increase.
But this is far behind a pattern of expansion at many other institutions - the University of Bristol's intake has grown by 22%, University College London by 36%, Queen Mary University of London by 46% and University of Surrey by 57%.
A spokeswoman from the University of Oxford said there were "no plans to expand overall undergraduate numbers or create new colleges".
Oxford's colleges already face "major challenges" over accommodation and the university runs many "initiatives which are expanding the number of students from under-represented backgrounds", said the spokeswoman.
The University of Cambridge also said it did not have any plans to significantly increase undergraduate numbers.
Prof Graham Virgo, pro-vice-chancellor for education, said: "Our biggest problem at Cambridge is convincing people they should apply and making it clear to them that they are welcome here.
"What message about inclusivity would be sent out by setting up a new college for this purpose?"
How university intakes have increased
- University of Surrey +57%
- University of Liverpool +50%
- Queen Mary University of London +46%
- University of Reading +43%
- Goldsmiths, University of London +38%
- University College London +36%
- University of Birmingham +28%
- University of Bristol +22%
Source: Hesa. Change in first-year undergraduate admissions 2012-13 to 2016-17.
Conor Ryan of the Sutton Trust social mobility charity suggests changing the admissions process so students apply after they know their exam results, rather than relying on predicted grades.
"Bright but poor students consistently have their grades underestimated," says Mr Ryan.
Anne-Marie Canning of King's College London says improving access for white working-class boys should be the "top priority", as they are the "most under-represented group in higher education".
Ending the "obsession" with academic degrees and providing a more balanced approach to vocational qualifications is recommended by Robert Halfon, Conservative chair of the Commons education select committee.
"Our labour market does not need an ever-growing supply of academic degrees. There are not the jobs available and for many graduates the return on their investment is paltry," says Mr Halfon.
Prof Kalwant Bhopal of the University of Birmingham believes "radical change is needed in higher education to support black and minority ethnic students".
She says there should be "mandatory unconscious bias training for staff".
Paul Clarke, of the mentoring charity Brightside, which co-produced the report with Hepi, highlighted the need to tackle the "precipitous decline in part-time students".
Many of these would have been from disadvantaged backgrounds, he says.
Mr Clarke also warned that much of the "heavy lifting" on recruiting students from disadvantaged backgrounds depended on a small number of universities.
The recommendations will be sent to the new director for fair access and participation, Chris Millward.
Mr Millward welcomed the report, saying: "Despite the progress made in access and participation for some groups, there are still wide gaps for mature students, for white males from the lowest income groups, and at the universities with the highest admissions requirements.
"And when students do enter higher education, certain groups also face real barriers to succeeding during and after their studies, particularly black and Asian students and those with disabilities," he said.