Short prison sentences serve no purpose and should be scrapped, according to the Howard League and the probation officers' union, Napo.
Napo said 74% of prisoners serving terms of less than 12 months were reconvicted within two years.
It wants money saved on prisons to go on supervising offenders in the community and setting up programmes to deal with drug abuse or violence.
The government said it was conducting a full assessment of sentencing policy.
Napo said offences given short custodial sentences could include actual bodily harm, theft, motoring offences and possession of indecent images.
Napo assistant general secretary Harry Fletcher said currently 55,000 people have prison sentences of six months or less, but rehabilitation was not possible and reoffending rates were "extremely high".
The cost to the taxpayer was at least £350m a year, he said.
Instead, he said the majority of these offenders could be supervised in the community on intensive programmes at a cost of between £50m and £60m a year.
"Not only would this option be cheaper, but the reconviction rates would be much lower," he said.
"It does seem extraordinary, therefore, that the government is actually cutting probation budgets, which is bound to lead to more, not less, custodial sentences, worse reconviction rates and therefore more victims," he added.
Frances Crook, director of the Howard League for Penal Reform, said short custodial sentences created more crime and were "a costly and wasteful response to complex human problems that needed solving".
"Upon arrival, prisoners on short sentences are handed their induction papers along with their release forms," she said.
"Nothing constructive can happen when a prisoner lies on a squalid bunk bed for three weeks."
She added that community sentences forced people to make amends for their wrongdoing.
The charity said short sentences were long enough to cause long-term damage, such as losing housing or having their children taken away. It believes prison sentences should be for those committing serious and dangerous crimes.
A Ministry of Justice spokesman said: "We will conduct a full assessment of sentencing policy to ensure that it is effective in deterring crime, protecting the public, punishing offenders and cutting re-offending."
Last year, the Prison Governors' Association passed a motion for the abolition of prison sentences of less than a year because they did not work and contributed to prison overcrowding.
At the time, the Magistrates' Association said governors had a duty to manage sentences and should not "meddle".
The Howard League is launching a research project with the Prison Governors' Association to find ways to reduce the need for short prison sentences.