Chancellor George Osborne says he is "shocked" that some of the UK's richest people have organised their finances so that they pay virtually no income tax.
The Daily Telegraph reported that a study by HM Revenue and Customs showed the very rich had reduced their average income tax rate to just 10%.
George Osborne said it was not right that the richest could legally arrange their tax affairs in such a way.
But Labour accused him of "synthetic shock", having cut the top tax rate.
He said he would take "further action" but did not outline any new proposals.
HM Revenue and Customs provided the chancellor with "anonymised copies" of the confidential tax returns submitted to the organisation by the UK's wealthiest people, the Telegraph reported.
He was not given the details of the individuals involved, but he said the returns he had seen had shown him the 20 biggest tax avoiders had legally reduced their income tax bills by a total of £145m in a year.
He told the newspaper: "I was shocked to see that some of the very wealthiest people in the country have organised their tax affairs - and to be fair it's within the tax laws - so that they were regularly paying virtually no income tax. And I don't think that's right."
"I'm talking about people right at the top. I'm talking about people with incomes of many millions of pounds a year.
"The general principle is that people should pay income tax and that includes people with the highest incomes," Mr Osborne added.
HMRC found the main methods used by people to reduce their bills was writing off business losses, offsetting the cost of business mortgages and borrowing on buy-to-let properties - all against their income tax bills.
Others took advantage of tax relief on charitable donations.
During last month's Budget the chancellor revealed that, from 2013, there would be a £50,000 cap on tax relief - or 25% of income if that was higher.
Although this was criticised by charities who feared they would lose out, on Tuesday Mr Osborne said the government was still examining ways of encouraging philanthropy and charitable giving.
Last month, Mr Osborne said he would cut the 50p top rate of income tax on earnings over £150,000 a year, which was introduced under the previous Labour government, to 45p from April 2013.
Businesses had complained it was deterring entrepreneurs and company bosses from investing in the UK and expanding their businesses and Mr Osborne said it was raising "next to nothing" and was damaging the economy.
But he said he would get five times as much from the wealthiest by other tax and anti-avoidance measures - and announced a rise in the threshold at which everyone pays income tax, which he said would leave millions of working people better off.
However, Labour's shadow financial secretary to the Treasury, Chris Leslie, told the BBC Mr Osborne had been shedding some "pretty heavy crocodile tears" and "synthetic shock" about tax avoidance.
"He seems surprised that the wealthiest in the country try to use tax loopholes. What is his reaction? It's to wave the white flag, put hands in the air, give up and cut that top rate of tax," he said.
"Either he's showing a serious lack of judgement or these are phoney words of concern when he's quite content to see the wealthiest get away without paying their fair share."
BBC political correspondent Iain Watson said the Telegraph interview suggested that the chancellor was trying to move the focus from his tax cut to the government's desire to tackle tax avoidance.