Ministers have said they feel "deeply ashamed" of low conviction rates for rape cases in England and Wales.
Speaking to the BBC, Justice Secretary Robert Buckland apologised to victims and promised to "do a lot better".
He also admitted budget cuts were partly to blame for convictions falling to a record low in recent years.
The government says it is now considering allowing victims to pre-record their evidence to spare them the trauma of a courtroom trial.
Its review also outlined plans to focus more on the suspect's behaviour - not the accuser's, and make sure phones taken away for evidence-gathering were returned within a day.
The review was met with mixed responses, with one rape charity saying it was a missed opportunity and failed to identify any big commitments that would radically and swiftly improve rape victims' experience.
Labour said the government had "failed victims of rape on every front" and its recommendations did not go far enough.
Speaking to BBC political editor Laura Kuenssberg, Mr Buckland said the report revealed "at all stages of the criminal justice process frankly a failure to deal with complaints made by many thousands of victims".
"The first thing I think I need to say is, sorry - it's not good enough."
In a foreword to the review, Mr Buckland, Home Secretary Priti Patel and Attorney General Michael Ellis, said: "The vast majority of victims do not see the crime against them charged and reach a court - one in two victims withdraw from rape investigations.
"These are trends of which we are deeply ashamed," they wrote. "Victims of rape are being failed."
They went on to say that most rapes were carried out by people known to the victim, making investigations very personal and intrusive for the victim.
The trauma and their experience afterwards led many to disengage from the criminal justice process, they said.
"We are not prepared to accept that rape is just 'too difficult' a crime to prosecute. We can, and must, do better," they added.
'It felt like it was my character being judged'
Katherine Araniello says she felt she wanted to take her own life after her rape case was dropped "practically on the eve of the trial" by the Crown Prosecution Service.
She says she was given an explanation as to why her case was dropped but it was only "very vague". "They didn't call me in to iron out any issues they might have at their end."
She says she felt like "it was my character" the Crown Prosecution Service were judging, not that of the alleged perpetrator, and her treatment by the CPS "really stung, and it still stings to this day".
"That lack of acknowledgment from the CPS on how to treat victims, highly traumatised victims as well, has left me really angry with the system," she says.
"They haven't learned their lessons."
Each year there are about 128,000 victims of rape and attempted rape but fewer than 20% of them report the crime to the police, according to the report. And just 1.6% of rapes result in someone being charged.
Emily Hunt, an independent adviser to the review and herself a rape victim, said the low prosecution rate was not because people made false allegations.
She cited Home Office research showing up to 3% of rape allegations could be false, which meant 97% of them were not.
"The reality is, in almost all cases, if someone says they've been raped, they have not made it up which makes the current situation all the more shocking and unacceptable," she said in the report.
What's the new plan?
Plans set out in the report include:
- Introducing better data extraction technology to reduce the time that victims are without their phones - with an aim to have them returned by police within 24 hours. Currently this process can take months causing distress for victims left phoneless at a time when they most need support from friends and family
- Putting greater emphasis on understanding a suspect's behaviour rather than focusing on a victim's credibility
- Sparing victims the trauma of attending a courtroom trial by videoing their cross-examination earlier in the process and away from the courtroom. A pilot will be trialled in several courts, with a wider rollout considered. This measure is already used for children and vulnerable victims and witnesses
The report - commissioned in March 2019 - also said the volume of cases going to court should return to "at least 2016 levels" and that regular "scorecards" would be published to monitor progress.
The Law Commission will also begin a review into "rape myths" and examine the use of a victims' sexual history as evidence.
Why are rape prosecutions falling?
There are many possible factors behind the fall in prosecutions.
The report argued it was due to "a strained relationship" between different parts of the system, lack of support for the victims and "an increase in invasive requests for their personal data".
Asked in his BBC interview whether government cuts to the legal system were a factor, Robert Buckland replied: "Like all parts of public service big choices were made in the last decade, because of the position that we all faced economically and that's, I think, self-evidently the case."
He said the government was now "seeking to make the necessary investment" but also insisted "it's about more than just money, it's about culture."
He argued that "for too long" rape allegations have focused on the credibility of the victim rather than the evidence.
"If your house is burgled people don't start asking 'why did you go out?'… all those assumptions I'm afraid are far too often not applied when it comes to sexual offences," he said.
In rape investigations, there has also been an increase in the amount of evidence to consider, often from phones and social media, making them more difficult for police, prosecutors and, potentially, victims.
Victims' Commissioner Dame Vera Baird has said some victims withdraw their complaints because "they cannot face the unwarranted and unacceptable intrusion into their privacy".
There have also been suggestions a secret conviction rate target, set by the CPS, led to prosecutors dropping weaker or more challenging cases.
In November 2019, it was revealed the CPS had introduced a target in 2016 that 60% of rape cases should end in a conviction.
What's the reaction been?
The victims' commissioner welcomed the review's proposals but said the government "has a mountain to climb if it is to restore victims' confidence". Dame Vera said it was "disheartening" that the pre-recording of video evidence was not being introduced more widely.
Labour's shadow solicitor general Ellie Reeves said the government's review had missed a "real opportunity" to improve the justice system, and accused the government of failing victims of rape "on every front by pushing the justice system to the brink of collapse through a decade of cuts to police, courts and the CPS".
She said "piecemeal pilots" and "tinkering" would not be enough to fix the failures.
Andrea Simon, director of the End Violence Against Women Coalition, said the current system was "in disarray".
She said it was welcome the government had admitted women have been failed, but women's groups felt "frustration and disappointment" over how long the measures in the review would take to be rolled out.
"Waiting a further two years to see some of these recommendations come to fruition is too long," she added.
Amelia Handy, policy lead for Rape Crisis England and Wales, said the review was "a missed opportunity".
"While there are individual elements of the government's report that are encouraging, it's hard to identify any big commitments that will radically and swiftly improve the experience of the justice system for victims and survivors," she said.
She called for "independent research into who does and doesn't report sexual offences to the police" and an urgent review into governance of the Crown Prosecution Service.
Sumanta Roy from Imkaan - an organisation tackling violence against women from ethnic minorities - called for ring-fenced funding "to address the historical under-funding of specialist independent support services for black, minoritised women".