Most two-bedroom flats in central London will be unaffordable for housing benefit claimants when reforms are introduced, says the head of Shelter.
Campbell Robb said it meant claimants faced getting into debt as they would have to subsidise rents themselves.
The cuts were being done too quickly and could change the "very nature" of London, he told the BBC.
The government wants to cap housing benefit at £400-a-week for the largest homes or £290-a-week for two-bed flats.
Ministers say the housing benefit bill has got out of control and people will still be able to claim a maximum of £21,000 a year - which they say is "more than the equivalent of what most working families have to spend on their housing costs".
But there have been concerns it will mean poorer people being forced out of expensive areas, like central London - and Labour says they will force a vote in Parliament on the plans.
Mr Robb referred to a study on the impact of housing benefit changes on London, commissioned by the charity from the University of Cambridge, which is due to be published this week.
It examined the impact of the cap, due to come in from April 2011, as well as other changes to the way rates are set.
He said early analysis had confirmed fears many London boroughs would become "largely unaffordable" for people on housing benefit from 2011, and a "significant further amount" would do so by 2016.
The research assumes rents will continue to rise by 3.6% a year - and does not include all proposed changes affecting private tenants on housing benefit.
But Mr Robb told the BBC: "It is absolutely clear from what they are saying, is that over the next few years a whole swath of London, a whole series of properties - 2-bed properties and bigger - will just become more unaffordable for those on housing benefit.
"In effect what will happen is the rents will be higher than the housing benefit that people get, so they would have to find their own money to meet the costs of rent before they have even started thinking about clothes and children and all those kinds of things."
He said he was concerned the cuts could change "the very nature" of central London, and other cities - and "could mean tens of thousands of households forced from the centre, creating concentrations of poverty and inequality".
And he said suggestions rents would come down if housing benefit was reduced were an "heroic assumption" when they had steadily risen over the past few decades.
Under the proposals, housing benefit payments would be capped at a maximum of £400 a week, with an upper limit of £290 per week for a two-bedroom flat.
The Department of Work and Pensions says nearly two-thirds of housing benefit tenants will be no worse off, or will face a gap of £10 or less a week between their rent and their benefits and the government has accused critics of "scaremongering".
However, official figures suggest that one in 10 households will face shortfall of £30 or more a week.
Earlier shadow health secretary John Healey told Sky his concern was that people would not be able to make ends meet and would get into debt - criticising the plans as "unwise and unfair".
However he also told the programme that he understood working people's concerns that some people were claiming the equivalent of £50,000-a-year salaries and said "those top-level benefits do need to be capped".
Work and Pensions Secretary Iain Duncan Smith said it showed Labour's position was in "chaos": "Labour have now added confusion to their breathtaking hypocrisy, opportunism and scaremongering."
This week, London Mayor Boris Johnson said he would "emphatically resist any attempt to recreate a London where the rich and poor cannot live together".
Prime Minister David Cameron said the government would still be "prepared to pay up to £20,000 a year" in housing benefit to families, adding: "I don't think it will be necessary for anybody to go without a home."
The Treasury believes the changes will trim £2bn from the £20bn housing annual housing benefit bill. Most two-bedroom flats in central London will be unaffordable for housing benefit claimants when reforms are introduced, says the head of Shelter.