John Prescott has accused bankers of "buggering up the economy".
The former Deputy Prime Minister said Labour could not be blamed for the recession as other European countries were suffering equally severe problems.
Mr Prescott made the comments in a debate with new Tory MP Zac Goldsmith on BBC Radio about housing policy.
He said the Tories would rather protect millionaires' gardens than build new houses but Mr Goldsmith said brownfield developments must be prioritised.
The two men clashed while debating government plans to give councils more powers to prevent so-called "garden-grabbing" by developers - where houses are built on gardens of existing properties snapped up for development.
Mr Goldsmith, a longstanding environmental campaigner, cited the state of the economy inherited when he rejected claims that the new government was not interested in building new homes.
He said: "If there are cuts in any sector that is a result of a legacy which, frankly, I think your party has a lot to answer for."
But Mr Prescott said people should blame the City, not the last government, for the scale of the deficit facing the country and its future impact on public services such as new housing.
"Don't be silly, that was an international problem. Why is it that US, Spain, Portugal, Greece have all got similar problems about debts.
"It might be down to some of your mates who are bankers. It is the bankers who buggered up the economy and everyone realised that started in America."
Mr Prescott said housebuilders opposed the government's plans but Mr Goldsmith argued the construction industry had a track record of "disproportionate influence" over where new houses were located.
"They have a direct vested interest in building, building, building. Frankly, their priority is not to protect the green spaces which people value.
"Our job is to ensure that those places which should be developed are developed."