The government says it wants a million homes built in England by 2020, as the scale of the housing crisis is revealed in a BBC Inside Out investigation.
The National Housing Federation estimated 974,000 homes were needed between 2011 and 2014.
But figures from 326 councils showed only 457,490 were built.
Housing minister Brandon Lewis said the government aimed to see one million new homes over this Parliament, but Labour said the Tories had failed on housing.
The federation said about 245,000 new homes were needed each year in England.
Gill Payne, director of policy and external affairs, said: "In some areas, there is a drastic shortage causing prices to soar, putting homes out of the reach of many people.
"Families and young people across the country are crying out for genuinely affordable homes so they can put roots down and achieve their dreams of owning a home.
"Skyrocketing rents and ballooning house prices are eating up more and more of people's wages and forcing people out of their local communities or into smaller, lower quality housing.
"We haven't built enough homes in this country for decades, and if the gap between the number of households forming and the number of new homes being built continues to grow, we are in danger of not being able to house our children."
Numerous factors have been blamed, ranging from planning procedures being too slow to developers sitting on large tracts of empty land instead of building on it.
In 2012, the government introduced changes to the National Planning Policy Framework, aimed at making the planning process simpler and quicker.
Some 240,000 planning applications have been given detailed permission in 2014 compared with 158,000 in 2011
Kingston upon Thames
But critics say the change has also made it easier for "inappropriate and unwanted" developments to progress.
They also say the rise may be simply a response to low numbers of applications caused by the 2008 financial crash.
A shortage of land has also been cited by homelessness charity Shelter, while criticism has been levelled at developers who build slowly rather than progressing quickly.
By keeping the number of new homes available at any one time low, the price of those houses can be kept high, said Matthew Pointon, property economist at Capital Economics.
"By building them more slowly it means they can maximise the value of their assets," he told the BBC earlier this year.
A shortage of skilled labour, a big drop in the number of councils building new homes and regulations restricting housing associations have also been blamed.
What might houses of the future look like?
What sort of homes will we need to build to meet housing shortages over the next 40 years?
Dr Philip Oldfield and the Sustainable Tall Buildings Design Lab at the University of Nottingham have been attempting to predict what new houses will look like in 2050.
From vertical villages and flat pack housing to eco homes, they have devised alternative schemes to solve future housing problems.
"We need the ability to provide housing en masse, there is a huge housing crisis at the moment," said Dr Mark Gillott from the university.
Mr Lewis said he wanted to make it "easier" for people to develop "especially on brownfield sites".
The minister added that further changes to planning rules would create a "principle of development" on brownfield sites to help councils determine applications quicker.
The Campaign to Protect Rural England estimates there are enough brownfield areas to accommodate 1.8 million homes.
Mr Lewis said: "Ultimately, it is up to local authorities to look at what their housing needs are and where they feel it is appropriate to build.
"I trust local people to get that right."
He said schemes such as Help to Buy, which sees the government assist buyers, should give developers the confidence that their builds will be sold.
Ms Payne said housing federations built 40% of all new homes across the country and had ambitions to "vastly increase this".
Labour's shadow housing minister John Healey said: "Ministers making big promises about homes in the future won't wash any more. People have seen five years of failure with Conservatives in charge of housing."
Shelter's chief executive Campbell Robb added: "We are past the time for another grand statement of ambition. To give ordinary families back the hope of a stable home, we need to see investment and a comprehensive plan that can actually get these homes built."
Inside Out is broadcast on BBC One on Monday, 21 September at 19:30 BST and nationwide for 30 days thereafter on the iPlayer