Cliff-top houses are losing a metre of their garden every year to erosion by the sea, a resident says.
A row of 12 houses perched on cliffs at Sidmouth in east Devon have all lost large parts of their gardens.
Martin McInerney, 82, has lost 20m (65ft) of his in the 20 years he has lived there and expects his home to be on the cliff edge in 60 years.
The Environment Agency says it is working to get funding for a scheme to reduce erosion.
Mr McInerney's garden is being lost in uneven stages.
The biggest single loss was a 10m (33ft) section and 14 months ago a 5m (16ft) chunk slipped into the sea about 30m (100ft) below.
Longshore drift - the movement of material along a coastline by waves - has taken protective sand and shingle westwards and exposed the cliffs say geologists.
Mr McInerney said he knew the cliffs were eroding when he and his wife Jo moved in "but it's happening faster than we thought".
"Fortunately we have got 60 metres left and given that I am in my 80s, I'm not too worried about it," he said.
"Although my grandchildren might not have the inheritance their parents would like them to have."
Despite the erosion Mr McInerney says he does not want to move.
"It's wonderful living here, light, airy and beautiful views," he said.
Residents are also concerned that low-lying Sidmouth town centre will be flooded unless the sea defences are repaired.
The Environment Agency said it was "supporting East Devon District Council and the local community" in making a case to get funding for a beach management scheme to protect the cliffs and the town.