Middle-income as well as low-income families are being held back by a "deep social mobility problem" in Britain, a report warns.
The Social Mobility Commission warns of many "treadmill families running harder and harder but standing still".
It points to an unfair education system, a two-tier labour market, a regionally imbalanced economy and unaffordable housing.
Prime Minister Theresa May has promised to tackle social inequalities.
The commission's State of the Nation 2016 report, laid before Parliament on Wednesday, says young families now can no longer expect to do as well as their parents' generation.
It highlights that home ownership is in sharp decline, particularly among young people.
The commission urges the government to commit to building three million homes over the next decade and to expand the sale of public sector land for new homes.
It should also allow targeted housebuilding on green-belt land and modify the starter home initiative to focus on households with average incomes.
By Hannah Richardson
BBC News education and social affairs reporter
Alan Milburn paints a depressingly familiar picture of a "them and us" Britain divided along the fault-lines of wealth and opportunity.
But the conclusion of his report - most families are now doing worse than their parents' generation - is staggering.
It means social mobility is going backwards not forwards, despite successive attempts to tackle it.
Phrases such as: "Huge swathes of Britain feel left behind," and: "The chickens have now come home to roost," pepper the report.
"So many families are struggling to get by, let alone get up," Mr Milburn said.
He highlights that of the 65 parts of the country identified by his commission as "social mobility coldspots", only three voted to remain in the EU.
Mr Milburn, himself immersed in issues surrounding the social mobility conundrum for six years, clearly feels the country's mood is now becoming dangerous.
And with a nod to 1930s Europe, he told reporters: "We know from the history of our own continent, when people feel they are losing out unfairly, the mood can turn ugly."
The commission notes that a child living in one of England's most disadvantaged areas is 27 times more likely to go to an inadequate school than a child in the most advantaged areas.
And young people from low-income homes are one-third more likely to drop out of education than better-off classmates with similar GCSEs.
The report says the government should have "as its core objective" the narrowing within the next decade of the GCSE attainment gap between poorer children and their better-off classmates by two-thirds.
It also urges the government to rethink its plans for more grammar schools and more academies.
The report says grammars are "at best, a distraction, and, at worst, a risk to efforts aimed at narrowing the significant social and geographical divides that bedevil England's school system".
The report notes a growing gap between the generations, warning that those born in the 1980s are the first post-war cohort not to start their working lives on higher salaries than their immediate predecessors.
It says the government must work with large employers, local councils and Local Enterprise Partnerships to bring high-quality job opportunities - backed by financial incentives - to Britain's social mobility "coldspots".
The report also calls for an end to unpaid internships, arguing that work placements lasting more than four weeks should be paid at the minimum wage.
The commission says the problem is not just social, but also geographical, with a widening divide between the big cities - particularly London - and the many towns and counties being "left behind economically and hollowed-out socially".
It says that in London, the number of top-end occupational jobs has increased by 700,000 in the past 10 years compared with just under 56,000 in the north-east of England.
The report warns: "The growing sense that we have become an us and them society - where a few unfairly hoard wealth and power - is deeply corrosive of our cohesion as a society.
"The EU referendum in June this year exposed deep divides that go well beyond the box that people crossed.
"Public concern - even anger - about issues of identity, immigration and inequality found a voice on 23 June and a target to aim at.
"Of the 65 parts of the country we identified as being social mobility coldspots - those with the poorest education and employment prospects - only three areas voted to remain in the EU."
Other concerns raised by the report include:
- Millions of workers - particularly women - are trapped in low pay
- Only one in eight children from low-income backgrounds is likely to become a high-income earner as an adult
- From the early years through to universities and the workplace, there is an entrenched and unbroken correlation between social class and success
- Despite some efforts to change the social make-up of the professions, only 4% of doctors, 6% of barristers and 11% of journalists are from working-class backgrounds.
The chair of the Social Mobility Commission, Alan Milburn, said the rungs on the social mobility ladder were growing further apart.
"It is becoming harder for this generation of struggling families to move up," he said.
"Whole sections of society and whole tracts of Britain feel left behind.
"As the EU referendum result showed, the public mood is sour and decision-makers have been far too slow to respond."
Theresa May took over as prime minister in July with a promise to lead a "one-nation" government that worked for all, not just the "privileged few".
Mr Milburn said the commission applauded her determination to further social progress but warned this was a "big ambition" and should be the "holy grail" of public policy.
A government spokeswoman said ministers would consider the recommendations in the report.
"We want to make this a country that works for everyone, not just the privileged few - whether it's education, jobs, or housing, this means giving families more control over their lives and doing more to help those who are just managing.
"As the prime minister said on the steps of Downing Street, this government is committed to fighting injustice wherever it arises and ensuring that everyone in our country has the opportunity to go as far as their talents will take them."
Shadow education secretary Angela Rayner said the report should make "sobering reading" for the Conservatives.
"Since they came to power in 2010, we have seen our country go backwards on the progress the Labour government made on social mobility," she said.
"The educational attainment gap between lower-income children and their wealthier classmates is getting bigger, and these children still have little chance of going into high-level professions.
"And under Theresa May we can expect more of the same: grammar schools for a select few, overcrowded classrooms for the many."
The report was published as a survey of 1,655 British adults for the commission found nearly half believed where people ended up in life was mainly down to their background and parentage.
Fewer than a third (29%) of people questioned thought Britain was a country where everyone could get as far as their talent and hard work would take them.
One-third of those surveyed said they could not save any money each month.
Join the conversation - find us on Facebook