Older people who want to downsize their homes should be helped to do so, "freeing up" billions of pounds of housing stock, a think tank says.
Those interested in downsizing are "sitting on £400bn of housing wealth" in England alone, according to Demos.
Its report, funded by the Home Builders Federation, says people aged over 60 own £1.28 trillion in housing stock.
It says more retirement homes are needed and if built, would help others move up the housing ladder.
In a survey of 1,500 people aged over 60 for Demos, 58% said they were interested in moving but felt restricted by a lack of suitable housing or a fear of moving to an unfamiliar environment.
And three-quarters of those who had homes with three bedrooms or more told researchers they wanted to downsize. The survey was carried out online across the UK in July, by Populus.
The report, called Top of the ladder, says the government could help older people by cutting their stamp duty and council tax.
Its author, Claudia Wood, deputy director of Demos, said: "Unlike in health or social care, the costs associated with overcoming the challenges of housing our ageing society are relatively small.
"The money for new housing is there already - locked up in over a trillion pounds worth of assets held by older people across the country.
"The majority of older people in three-, four- and five-bedroom homes want to downsize.
"Overcoming planning barriers to supply to meet this demand would benefit the economy, younger families stuck on the housing ladder and older people themselves."
The report says there is a "chronic undersupply" of specialist retirement housing, especially for "very old" people, whom it defines as those aged over 80.
This group, it says, is increasing more rapidly than other groups, as society ages.
More than two-thirds of this group have a long-term illness or disability, compared with a third of those aged between 65 to 74, it reports.
"This increasingly old population may well need housing that offers care and support services on site," the report says.
Michelle Mitchell, from Age UK, said: "We certainly badly need more retirement housing in this country and this could help to free up more family-sized properties.
"But we don't just need more retirement homes, we need them to be of consistently high quality, affordable and attractive, and in the right places."
Baroness Sally Greengross, head of the International Longevity Centre-UK think tank, said people should "fight the ageist attitudes that blame older people for hanging onto their homes" - and find "active ways to build the retirement housing market in this country".
"For too long we have thought of older people as a homogenous group, whose housing needs are one-size-fits-all," she said.
"It is vital that we change our thinking, and give people what they really want from retirement housing, rather than what we imagine they want based on outdated stereotyping."
'Plan for ageing nation'
The government says homes for the elderly form part of its plans to invest in and encourage the building of more affordable homes.
It recently announced it was going to give a discount of council tax for annexes for grandparents or other family members.
Housing Minister Mark Prisk said: "It is essential that we build more homes or suitable accommodation for older people.
"That's why new planning guidance makes clear that councils must plan positively for an ageing nation and ensure that enough suitable homes such as bungalows or assisted housing are available in their area.
"This is part of our wider efforts to get Britain building, investing billions towards delivering the fastest rate of affordable housebuilding for two decades - including £315m for homes designed to meet the needs of older people.
"Our plans to remove the family tax penalty on annexes and home improvements will also help more older people live independently."