Banks operating in the UK will be hit with a levy in a move set to raise more than £8bn over four years, the chancellor has announced
The levy is part of a joint move between the UK, France and Germany.
George Osborne said the financial crisis began in banking so it was fair that banks contributed to the recovery.
But the British Bankers' Association warned that the levy must not harm job creation nor weaken the UK's strength in financial services.
The levy, which will not affect smaller banks and building societies, will be brought in from January 2011.
The UK operations of foreign banks will also have to pay the levy.
Mr Osborne said: "This was a crisis that started in the banking sector and the failures of the banks imposed a huge cost on the rest of society.
"So it is fair and right that in future banks should make a more appropriate contribution which reflects the many risks that they generate."
According to details in the full Budget document the levy will be set at 0.04% in the first year, generating about £1.15bn.
The levy will then rise to 0.07%, raising an estimated £2.3bn in 2012-13, £2.5bn in 2013-14, and £2.4bn in 2014-15.
Critics of slapping a levy on banks have said that it cannot work without international co-ordination.
The simultaneous move by the UK, France, and Germany is designed to help prevent banks avoid the levy by shifting operations between countries.
In the US, President Barack Obama wants a 0.15% levy on banks' net liabilities, raising up to $117bn over 10 years. But the proposal is bogged down in Congress.
A joint statement issued in London, Berlin and Paris said the three countries wanted financial institutions to make "a fair and substantial contribution".
"The specific design of each [levy] may differ to reflect our different domestic circumstances and tax systems, but the level of the levy will take into consideration the need to ensure a level playing field," the statement said.
Germany "will present draft legislation" for a levy in the summer and France "will present the details of its bank tax in the coming budget" later this year, the statement said.
And it added that the three countries "look forward to discussing these proposals further with international partners" at the G20 summit in Canada this weekend.
In the weeks before the election there was speculation that the UK chancellor was looking to raise about £3bn a year from a levy, and possibly even as much as £5bn.
Robert Peston, the BBC's business editor, said: "Some may say [the banks] got off lightly, with a levy to raise a maximum of £2.5bn [a year] by 2013-14.
"It will be a tax on the total size of their balance sheets, minus their insured retail deposits and their capital, and with a lower rate applicable to longer-term wholesale funding."
On this basis, he believes, banks Lloyds TSB and Royal Bank of Scotland are likely to pay the largest chunks of the levy, followed by Barclays and then HSBC.
The British Bankers' Association (BBA) said it accepted the reasons for a levy, but warned that it must not be allowed to hurt the competitiveness of the UK as a financial centre.
"The banking industry fully understands the part it must play in helping the UK's economic recovery," the BBA said.
"The banks are committed to working with the government to ensure new bank levies balance tax raising objectives with the need to keep the recovery moving".
However, the BAA added: "The UK is a trading nation and we must ensure bank taxes do not hurt our national interests or provide an unfair advantage for other businesses operating here.
"We are a large financial centre and a great many jobs are created here as a result. So bank levies need to be co-ordinated internationally: they must not prevent the industry in the UK from being able to compete."