Wealthy elderly people who do not need benefits to help with fuel bills, TV licences or free travel should return the money, the work and pensions secretary says.
Iain Duncan Smith told the Sunday Telegraph he would "encourage" people who do not need such financial support "to hand it back".
But the decision whether or not to do so was up to them, he added.
Deputy PM Nick Clegg said the suggestion did "not make sense".
Prime Minister David Cameron has already said the benefits would be protected for the entirety of this Parliament, including for the year 2015-16.
A general election will be held in 2015, and Mr Duncan Smith said there were no plans to means test to exclude richer pensioners.
He told the paper: "I have no idea what we will put into the manifesto... I have no indication of change.
"It's fair to say that [pensioners] are more vulnerable than others and we need to be very careful about what and when we do things, if we ever do."
Among the benefits is the winter fuel allowance of between £100 and £300 tax-free to help people who have reached the state pension age with heating bills.
A free TV licence, worth £145.50 for a colour set, is provided for people aged 75 or over, while those aged over 60 can get free NHS prescriptions.
Eligible older people are entitled to free off-peak travel on local buses anywhere in England when they reach the state pension age. Schemes in Scotland, Wales and Northern Ireland allow people aged over 60 to apply for free bus travel.
A Department for Work and Pensions spokeswoman said people have always been able to contact the department if they no longer want to receive a benefit.
Mr Clegg said his Liberal Democrat party and coalition partner the Conservatives were at odds over pensioner welfare reform: "I think we should grasp this nettle. The Conservatives don't want to do so. That is a difference of approach."
He added: "When money is tight, you've got to have the right priorities in tough times. I think it's right to ask very wealthy, maybe multi-millionaire, pensioners to make sacrifices, just as we're asking families on lower incomes."
Shadow communities secretary Hilary Benn said: "I don't agree with him (Mr Duncan Smith).
"The fact is we have certain universal benefits in our society, people have worked hard, they've paid their taxes and they get that, in this case, when they have retired."
Entrepreneur Lord Sugar and nightclub owner Peter Stringfellow have been among those who have previously complained they have not been able to opt out of receiving the winter fuel allowance.
Former Conservative Chancellor Ken Clarke said: "Well, you can't hand it back to the government, I don't think it's a system for doing that.
"I think every pensioner and retired person, like myself, obviously has to make up their own mind about whether they really need it and whether they're going to give it to some worthwhile cause.
"No doubt, most pensioners who are reasonably prosperous give quite a lot of money to charity and worthwhile causes in any event."
Charity Age UK said encouraging wealthy people to give up the benefits could have a knock-on effect.
"It is open to anyone to decide not to make use of these benefits but when it is suggested that 'wealthier pensioners' should choose to forego them our worry is that some who are badly in need of extra help will feel less inclined to take it," director of external affairs Caroline Abrahams said.
"Older people on very low incomes sometimes minimise their own difficulties and refer to others they know who are worse off than they are."
Meanwhile, the introduction of a new system of benefits for working age people begins on Monday - and marks the biggest overhaul of the benefits system since the 1940s.
Mr Duncan Smith said the new universal credit system was being implemented over four years because "I want to get these things right".
Under the new universal credit, six working-age benefits - income-based jobseeker's allowance, income-related employment and support allowance, income support, child tax credit, working tax credit and housing benefit - will be merged into one.
He said: "We want to say to people, you're claiming unemployment benefit but you're actually in work paid for by the state: you're in work to find work. That's your job from now on: to find work."
Universal credit starts on a limited basis on Monday for new claimants, who are single, who live in a small number of postcode areas in Ashton-under-Lyne in Tameside, Greater Manchester.
Trials in three more areas - Oldham, Wigan and Warrington - will begin in July.
From October, more claimants will move on to universal credit as and when they have a significant change of circumstances, such as starting a new job or when a child is born.
From April 2014 until October 2017, the rest of those affected in England, Scotland and Wales will be moved on to universal credit in stages. It will start in Northern Ireland from April 2014.