More than 100 LGBT organisations and celebrities are urging the government to reform UK gender identity laws, the BBC has learned.
Following a public consultation - which had 100,000 responses - changes were promised, but the government has continually missed its own deadlines.
The Government Equalities Office has insisted any potential reforms must be carefully planned.
They must have a positive impact on the trans community in the UK, it added.
But, in a letter addressed to Prime Minister Boris Johnson, LGBT groups said the UK was "lagging behind" other countries "in terms of legal equality for trans and non-binary people".
The letter - seen by the BBC - has been signed by more than 100 charities, campaigners and activists, including Stonewall CEO Ruth Hunt, musicians Olly Alexander and Jade Thirlwall, and trans activist Munroe Bergdorf.
Ashleigh Talbot, a transgender woman, says the current process to get legal recognition is too negative.
"It's an extremely bureaucratic process," said Ms Talbot.
"You have to gather years of evidence from employment records, identity documents and bank statements, and send all of that information to a panel who you don't actually meet.
"These people will then make a decision about whether you are 'trans enough' to be covered by the law."
Ms Talbot added that for any other community to be required to prove something to a panel in order to be granted full rights would be an "absolute outrage".
She continued: "I know people who have had their application turned down because of a perceived fault with the evidence - they were so humiliated.
"The stress and the emotional toll that it takes on members of the community, simply to have legal recognition, is all extremely damaging."
The Gender Recognition Act (GRA) sets out the legal process by which a person can change their gender.
It was last updated in 2004, and was the first piece of legislation that officially recognised transgender people but, since then, the way people identify has come a long way.
Currently, the gender law requires a medical diagnosis, or presentation of evidence, for trans people to get their identity legally recognised. It also does not recognise non-binary identities.
Some argue reforms could put people in danger of others fraudulently changing their legal gender, as well as lead to trans individuals regretting their choice.
Dr Nicola Williams, director of Fair Play For Women - which campaigns for the sex-based rights of women and girls - said GRA reform would mean any male could change their birth certificate to say they were born female.
"Women have a lawful right to exclude males from female-only spaces when it's necessary for privacy, safety and fairness," she said.
It would make women's existing legal rights "unworkable", she added, and that advancing rights for one group "at the expense of another vulnerable group" is wrong.
Jamie Windust, an LGBT and non-binary activist who does not identify as a man or woman, says things need to change.
"Non-binary people like myself are not even given the option to change our gender identity through the GRA," they said.
"This is not only invalidating, but having your own government not allow you to just exist is really painful.
"We need reform now rather than later because it truly is a situation that is juggling with people's lives and delaying will cost lives."
In 2018 the government launched a public consultation in order to understand gender in more detail and explore ways in which the system could be improved.
It acknowledged that many trans people find the current requirements "too bureaucratic, expensive and intrusive".
The consultation closed in October 2018 but the government has yet to respond.
A response was promised "by spring 2019", and then "before Parliament's summer recess", but now an update is expected by the end of 2019.
The letter to the PM was started by Paul Martin OBE, chief executive of LGBT Foundation, who said: "We have come together to reiterate how critical these reforms are to removing the barriers that trans and non-binary people face every day.
"By introducing new legislation to allow changes to trans and non-binary people's birth certificates without a judgement panel, high fees, doctors letters and evidence, the biggest change is that their dignity and rights are respected.
"Further delays are not acceptable and will have a significant and negative impact on trans and non-binary people."