Banks are trying to maximise short-term profit at the expense of customers, the Bank of England governor has said.
In a Daily Telegraph interview, Mervyn King questioned the bonus system and warned that failure to reform the sector could result in another financial crisis.
BBC correspondents say the remarks are significant because the Bank of England is taking over regulation.
Banking chiefs said they "respected" Mr King but disagreed with his comments.
BBC business editor Robert Peston said the interview showed Mr King had "little respect" for the banks.
His remarks come weeks after Chancellor George Osborne signed Project Merlin.
The deal agreed that in return for the banks lending more money and showing restraint on bonuses, the government would not take any more action on pay and profits.
Mr King said that, over the past two decades, too many people in financial services had thought "if it's possible to make money out of gullible or unsuspecting customers, that's perfectly acceptable".
He said: "Why do banks in general want to pay bonuses? It's because they live in a 'too big to fail' world in which the state will bail them out on the downside.
"We've not yet solved the 'too big to fail' or, as I prefer to call it, the 'too important to fail' problem.
"The concept of being too important to fail should have no place in a market economy."
He also said there was "too much weight put on the importance and value of takeovers" and that "it doesn't make sense to destroy a company with reputation" in the hunt for short-term profits.
He went on to draw a contrast between manufacturing companies, which he said largely cared about their workforces, customers and products, and the banks.
"There's a different attitude towards customers. Small and medium firms really notice this: they miss the people they know," he said.
Chief executive of the British Bankers' Association Angela Knight said some in the industry had "got it badly wrong during the [financial] crisis" but since then the industry had "reformed radically".
"We entirely agree that no bank should believe it can fall back on the taxpayer. The changes from top to bottom within the industry have ensured the risks are well controlled," she said.
Angela Knight added that they hoped to discuss the issues with Mr King, adding: "I sincerely hope he is as committed as we are to forming a good relationship [when the Bank of England takes over regulation]."
Deputy Prime Minister Nick Clegg said: "I understand why people are angry when they hear about the super-sized salaries and bumper bonuses awarded to top bankers. I am too.
"I agree with Mervyn King today that the job of making our banks safe and responsible is not yet complete," he told delegates at the Scottish Liberal Democrat spring conference in Perth.
BBC business editor Robert Peston said the fact that Mr King had, in the interview, "vilified" the banks was significant.
He wrote in his BBC blog: "He will be, without any near challenger, the most important person in the lives of our banks. So it matters that he appears to have so little respect for them."
He added that although it would be "unhealthy" for there to be a "chummy relationship" between the person keeping the banks in order and the banks themselves, banking chiefs have alleged Mr King doesn't understand what they do.
"Some of the things that board members of banks have said to me about Mr King are unprintable," said our correspondent.
Treasury minister Justine Greening told BBC News that what Mr King had said would resonate with the government and with "people".
"We have said ourselves that we think we need to improve regulation in the banking sector - it's one of the reasons why we've set up the International Banking Commission. That is looking at a whole range of aspects of how the financial services industry works."
She said that what the governor was saying was that "unlike in the past... we have got to get this right".
"The banking commission will be reporting later this year, we will be able to legislate next year, and Mervyn King and the Bank of England will be playing a much bigger role in regulating the industry - of course he is setting out, clearly, some of his priorities."
On Project Merlin she said people would "recognise" Mr King's comments about the banking industry in terms of how they deal with consumers.
This was one of the reasons the government had announced the establishment of a new authority to look at consumer protection [Consumer Protection and Markets Authority] in more detail, she added.
Economist Tim Congdon, who was a member of the independent Treasury panel in the 1990s, said he found it "incredible" that Mr King was attacking the structure of the industry he had been involved with for 20 years.
"We have a very successful industry, it competes internationally, we want it to do well, it employs hundreds of thousands of people, and I find the things that Mr King has been saying extraordinary," he said.
He added that Mr King, and others, caused the financial crisis "not least because they undermined the ability of British banks to fund themselves in international markets".
"If you criticise the banks, you reduce their credibility and then people worry about them. The important thing for the governor of the Bank of England is to support the banks and to help them."
The newspaper interview comes as Chancellor George Osborne prepares to tell his party's spring conference that the Budget later this month will be "unashamedly pro-growth".
Mr Osborne will announce that 10 new enterprise zones will be set up in England, in his Budget on 23 March. They will aim to regenerate growth in areas hit hard by the downturn.