Treasury minister David Gauke has said it is "morally wrong" to pay tradesmen such as plumbers, builders and cleaners in cash in the hope of avoiding tax.
He argued the practice came at "a big cost" to the Treasury and meant other people had to pay more.
But Treasury sources stressed Mr Gauke was answering a specific question rather than proposing a policy change.
Labour said the government should be focusing on "clamping down" on "large-scale tax avoidance".
Asked if he ever had paid cash to tradesmen to reduce his costs, London Mayor Boris Johnson said: "I've certainly paid a lot of cash in hand."
In response to the same question, Local Government secretary Eric Pickles said: "Certainly not."
And Culture Secretary Jeremy Hunt said: "David Gauke was saying that tax-dodging is wrong and personally I haven't dodged tax."
The government is estimated to lose about £2bn each year to the black economy as tradesmen fail to pay VAT or income tax by not declaring payments and keeping them "off the books".
Mr Gauke told BBC Two's Newsnight there was nothing wrong with paying in cash, but doing so actively to avoid tax was wrong.
Mr Gauke said: "When a tradesman says, 'Here's a 10%, a 20% discount on your bill if you pay me cash in hand' that is facilitating the hidden economy. That's as big a problem in terms of loss to the Exchequer as tax avoidance. Revenue is not being paid as it should be paid."
He added that he had "never said to a tradesman, 'If I pay you cash, can I get a discount?'"
But asked if he thought any ministerial colleagues had done so, Mr Gauke replied: "I don't know, but if people do do that they have to do so with the recognition that means taxes will be higher for the rest."
The Daily Telegraph quotes Mr Gauke as saying: "Getting a discount with your plumber by paying cash in hand is something that is a big cost to the Revenue and means others have to pay more in tax.
"I think it is morally wrong. It is illegal for the plumber but it is pretty implicit in those circumstances that there is a reason why there is a discount for cash."
But Tariq Dag Khan, of the tradesman recommendation website Rated People, said: "David Gauke's comments that it is morally wrong to pay tradesmen in cash do little to help tradesmen who are struggling in a difficult economic climate.
"For the reality is that there is little or no alternative to cash payments for many tradesmen, and criticising the whole industry belies a misunderstanding of the situation many customers and tradesmen are in.
"There is a great deal of trust involved when hiring a tradesman for both the tradesman and the customer and therefore cheque payments do not provide a viable alternative especially when, if the cheque bounces, the tradesmen could be dangerously out of pocket and in some circumstances forced out of business as a result."
Labour leader Ed Miliband, on a visit to Paris to meet French president Francois Hollande, said: "What I say is that the job of government is to pass the right laws to clamp down on tax avoidance - that's the most important thing of all.
"What I will be saying to the government is that they should be clamping down on the large-scale tax avoidance which has been revealed in the past few days and I think that's what people want to see from the government."
Last month Prime Minister David Cameron condemned the use in the past by comedian Jimmy Carr of a Jersey-based scheme as "morally wrong" and in his Budget speech Chancellor George Osborne described tax avoidance as "morally repugnant".
Mr Gauke was interviewed after he outlined proposals to tackle "aggressive" tax avoidance schemes, promising the government would force "cowboy" financial firms to disclose the names of people using them.