Carers looking after older people should be given statutory LGBT training to help tackle homophobia in the sector, the charity Stonewall has said.
Discrimination is "alive and kicking," with many older people fearing abuse from care workers, one man told the Victoria Derbyshire programme.
Currently, there is a code of conduct but all training is voluntary.
The government said making training mandatory was not possible as care providers were private employers.
There are an estimated 1 million LGBT people over the age of 55 in the UK. They are more likely than their heterosexual peers to be single and to live alone, and less likely to have children or see family members, according to YouGov research.
They are nearly twice as likely to expect to rely on health and social care services and paid help.
The industry's regulator, the CQC, said care workers were expected to have a care certificate, but there was no mandatory training in the industry.
'Right to be concerned'
Paul Webley, a co-ordinator at Opening Doors London, part of Age UK, has worked with more than 200 LGBT clients over the past five years. He says older people are right to be concerned.
"I absolutely know that homophobia is alive and kicking in the care industry.
"I've met many people who tell me stories of carers who, when they realise they're a gay woman or a gay man, don't want to be washing them."
Research carried out by LGBT rights charity Stonewall in 2011 found that half of those surveyed would not feel comfortable being "out" to care home staff and one in three would not be comfortable being out to hospital staff, a paid carer, social workers or to their housing provider.
Stonewall's campaign director Paul Twocock said a one-size-fits-all approach to equality and diversity had created gaps in how LGBT patients of all ages were cared for.
"Health and social care staff need better equality and diversity training to understand and meet the needs of all LGBT patients and service users, including older people," he said.
It is difficult to get official figures on the scale of the issue as the main regulatory bodies - the Local Government and Social Care Ombudsman and the Parliamentary and Health Service Ombudsman - both said they did not record complaints specifically concerning homophobia.
"One carer didn't know about me and actually turned round in conversation afterwards and said she'd heard about people like me, that in her country 'they would put a tyre around you and set fire to you'" says Andrew Van-Hove, 67.
He began transitioning from female in 1968, and 50 years later, says is he scared to have a carer help wash him because of their potential response.
"I didn't want the carer to see me, to see my naked body. And I didn't want somebody coming into my home and giving me a wash and then going out there and telling the rest of the world what I was," he explains.
Tina Wathern, who works for Stonewall Housing which represents gay people's housing needs in England, said LGBT training for care staff should be mandatory.
"It's not seen as being significant, it's often more about medical requirements, there's not enough thought being given to that relationship between the carer and the person being cared for," she said.
Christies Care, which has more than 400 clients and is rated outstanding by the CQC, says its staff have training around discrimination.
Managing director Freddy Gathorne-Hardy said: "We treat everyone as an individual and we treat people the way they want to be treated. We ask our clients how we can improve our service and this is anonymous. We ask them all the time… no-one has raised that in all our years."
Skills for Care, a charity which works to improve social care, is currently conducting a consultation for the government to help shape a strategy to improve the skills of the social care workforce.
It says it is an opportunity for anyone to express their views on what should be in the strategy, including views on what training should be mandatory.
The Department of Health and Social Care said it was "completely unacceptable" for anyone receiving social care to be discriminated against because of their sexual orientation.
It said it could not impose mandatory training on private employers but worked with Skills for Care to create training products and resources so care providers can implement their own training.
"We want everyone receiving care to be treated with the dignity and respect they deserve - that's why the regulator holds care providers to account to ensure they meet the fundamental standards we expect," a spokesman said.
But Mr Webley said the treatment by some carers made him "very angry".
"They've spent enough of their lives suffering all the years of homophobia and abuse and being arrested and just being made to feel like second-class citizens in a country where they were brought up and lived and paid their taxes and everything else. I swear that I hope I never go into care."
Watch the BBC's Victoria Derbyshire programme on weekdays between 09:00 and 11:00 on BBC Two and the BBC News channel.