Ministers are being warned that a proposal to scrap prison sentences of six months or less will only work if there is more investment in the probation services.
The government says this approach in England and Wales' prisons could reduce overcrowding and re-offending.
It is thought it would free up thousands of prison places.
But the probation officers' union says the partial privatisation of the service needs to be reversed.
About 30,000 offenders - including burglars and most shoplifters - could avoid jail every year under the Ministry of Justice plans.
Sentences for violent and sexual crimes would not be affected by the proposed change, which has been welcomed by prison reform charities as well as trade union Napo.
Ian Lawrence, Napo's general secretary, said while he supported the concept, there was "one important caveat".
He told the BBC the part-privatisation over the past four years had "had a massive impact on service delivery and the ability of our skilled professionals to do exactly what they should be doing".
"Levels of supervision, in many regards, are unsatisfactory," he said. "It's not conducive to proper rehabilitation. It's not cost-effective for the taxpayer.
"And what we need to see is more provision in the community for people to be given the opportunity to turn their lives around."
Offenders could face sanctions such as community sentences if the proposal goes ahead, but no alternative penalties have been confirmed.
Ministers argue that short jail terms are less effective at cutting reoffending than community penalties.
Prisons minister Rory Stewart told the Daily Telegraph that such sentences were "long enough to damage you and not long enough to heal you".
Arguing for the need for reform, Mr Stewart said: "You bring somebody in for three or four weeks, they lose their house, their job, their family, their reputation.
"They come (into prison), they meet a lot of interesting characters (to put it politely) and then you whap them on to the streets again.
"The public are safer if we have a good community sentence... and it will relieve a lot of pressure on prisons."
Frances Crook, chief executive of the Howard League for Penal Reform, told the BBC: "We should not be using prison for these people. We should be turning their lives around and giving them support - dealing with their drug addiction, their homelessness - and getting them to make amends for the wrong they have done.
"That's really good for victims, it's good for the taxpayer and it will ease pressure on prisons."
The Prison Reform Trust, which has previously called for a presumption against short prison sentences, welcomed the new proposals.
Peter Dawson, the charity's director, told the Telegraph: "Ministers should be congratulated for having the political courage to start the debate."
In Scotland, a presumption against prison sentences of less than three months is already in place and is due to be extended to 12 months.
More than half of the 86,275 offenders sentenced to immediate custody in England and Wales in 2017 were handed sentences of six months or less, according to a Parliamentary response from Mr Stewart last month.
The prison population has doubled in England and Wales since the early 1990s, rising from about 40,000 to more than 80,000 in 2018, official figures show.
Almost two-thirds of prisoners released after sentences of less than 12 months reoffend within a year.
What impact do short prison sentences have? Tell us about your experiences by emailing email@example.com
Please include a contact number if you are willing to speak to a BBC journalist. You can also contact us in the following ways: