The number of women being sent to prison should be halved, Justice Minister Simon Hughes has said.
Female offenders are a "special case" and should be treated differently to men because many had been victims themselves, he told BBC Radio 5 live Breakfast.
There are currently around 3,800 women in prison in England and Wales.
Further government measures to help stop women ending up in custody are expected to be announced later.
"There are so many women who ought not to be in prison. About half ought not to be there at all," the minister said.
"I met a woman in her 20s the other day who clearly ought to be sectioned. Her problem is a health problem, not a criminality one. Prisons shouldn't have to cope with that."
When asked why female offenders should be treated differently to men Mr Hughes said: "Women are a special case for very good, evidenced reasons. Firstly, many more women who go to prison have themselves been victims. They've often been abused or in violent partnerships.
"Secondly, many more women have caring responsibilities than men do."
Rachel Burden, BBC Radio 5 Live breakfast presenter
Plans to reform the way female offenders are dealt with - hauling together the various agencies which can sometimes be at odds with each other - have been in the pipeline for a while.
Now up and running in Greater Manchester, the various agencies - police, magistrates, women's centres, prisons, probation - intervene at the point of arrest, sentencing or release to offer alternatives to custody, or to make sure women don't return to prison.
Those involved say it's helping to reduce reoffending. Justice Minister Simon Hughes wants other English cities and regions to adopt the model. But the question many are asking now is - if it's so successful for women, why can't it be applied to male offenders too?
Mr Hughes said very few women were inside for violent offences.
"There are some women who do terrible things and deserve to be locked up for a very long time," he said. "My concern is for those who are not a danger to society, who have become caught by a system which then does not help them out of it."
Mr Hughes was speaking ahead of an announcement expected to say that a scheme in Manchester to stop women ending up in prison will be rolled out to six other areas in England.