The mental health charity Mind says it is signposting people to street drug charities to help them withdraw from antidepressants because of the lack of alternatives available. Those affected can experience debilitating symptoms.
"Within a couple of days of coming off, it was overwhelming - agitation, anxiety, akathisia [restlessness], just restlessness, can't sleep, suicidal ideations, all that stuff going on very quickly," Stuart Bryan tells the BBC's Victoria Derbyshire programme.
The 48-year-old has been taking anti-depressants on and off for more than two decades.
"The withdrawals are far worse than the original depression, for me and so many other people."
Stuart has tried to stop more than 10 times, but has struggled with what he calls his withdrawal "hell" - and has now had to stop working.
He says doctors have advised him to take anything between "a few weeks" to three months to slowly stop using the drugs.
But he believes people coming off anti-depressants are being "abandoned by the system".
While antidepressants are not addictive, just over half of those who stop or reduce their dosage experience withdrawal symptoms, according to one review of 24 studies last year.
The mental health charity Mind's head of information Stephen Buckley says it is having to signpost patients to street-drug charities, even though they have been prescribed the drugs on the NHS.
Street-drug charities usually help those misusing alcohol and illegally-obtained drugs.
"The process of withdrawing from something like a street drug and withdrawing from a medicine is not that dissimilar," he says.
"It's not really well understood how long and how difficult a process coming off anti-depressants can be.
"If you've been prescribed and been on a medicine for about six months, we might think that it'll take you another six months to successfully reduce and come off that medicine.
"It's not a process that for most people can be done really quickly."
The National Institute for Health and Care Excellence advises doctors to gradually reduce anti-depressant dosages over a four-week period.
But, it adds, some patients will need longer to adjust and updated guidelines are expected to come out later this year.
The Royal College of Psychiatrists, meanwhile, says the fact some patients who stop or reduce their dosage experience significant withdrawal symptoms "needs greater recognition" within the health service.
Prescriptions for anti-depressants in the UK have doubled in the past decade.
Health Secretary Matt Hancock told the BBC's Victoria Derbyshire programme in a statement that he was "determined everyone has access to excellent mental health care".
"Anti-depressants have a place but they must be used with caution," he said.
"Our world-class medical staff have the skills and training to ensure they can diagnose accurately and decide on the best treatment plans."
Back in Worcester, Stuart is getting ready to welcome other people withdrawing from prescription drugs at his monthly support group.
He decided to set it up after finding it difficult to get support when he began trying to withdraw.
"I made a self-referral to the local substance misuse charity," he says, "but then I was told it was only for people that was abusing the medication, which I wasn't, so there was no service available."
"It was really an acute need, and those needs aren't being met by services.
"It's left for people to do it themselves."