Most people want employers to lose the right to tell people when to retire, according to a survey by the charity, Age UK.
It says the government should scrap forced retirement immediately, currently set at 65 years for both men and women.
The survey showed more than two thirds thought the principle of forced retirement was wrong.
Age UK is asking the government to make clear its plans for the retirement age.
Under the current system, employers can tell any member of staff to leave work once they pass their 65th birthday, whether the individual wants to stop work or not.
One who did not want to retire was John White, a 69 year-old postman.
He was forced to leave work in July last year, despite the fact he loved his job and was good at it - a fact confirmed by his manager.
He points out that the higher-paid professions, such as politicians, doctors and judges, can keep on working past 65 - the age at which the state pension is paid to men.
'Speculation and confusion'
Women had been allowed to draw theirs from age 60, but that is now slowly changing to catch up with men.
The coalition government has said it intends to scrap the forced retirement age, but has given no guidance as to when it plans to do this.
Age UK, formerly known as Help the Aged and Age Concern, is calling on ministers to end what it calls "speculation and confusion" among employers and workers and set a date to end forced retirement.
The organisation says tens of thousands of workers were forced to retire last year.
"Forcing people in later life out of the labour market when they want to work, save for their pensions and pay taxes, is nonsense," Michelle Mitchell, charity director of Age UK, said.
"It makes a mockery of the Government's plans to help people work longer."
Age UK said Tuesday's Budget offered ministers the opportunity to say "loud and clear" when they were going to abolish "forced" retirement.