Officers are not always "sensitive and empathetic to the rape victim's account", Met Police Commissioner Sir Bernard Hogan-Howe has said.
At the launch an independent review of the investigation of rape, Sir Bernard said the force needed to "drastically improve" how it handled rape cases.
More than 80% of sexual offences victims do not come forward, he added.
The review, part of a national action plan, will be led by ex-Lord Advocate of Scotland Dame Elish Angiolini.
The outcome will be published in February.
Sir Bernard said: "For a while, I've been concerned about how we, the police, approach and deal with sexual offences.
"This is our brave attempt to try to resolve it," he added.
He said people did not seem to "trust the criminal justice system to give them the assistance they need" and about 80% of those who come forward to report were "vulnerable" through drink, drugs or psychiatric illness.
"My concern has been whether or not the investigators are always as sensitive and empathetic to the victim's account, as they should be."
Cases like that of London cab driver John Worboys, who was jailed indefinitely after being linked to attacks on more than 100 women, show "we need to drastically improve how we deal with criminal investigations into rape", he added.
The commissioner added that a stricter approach to rape allegations meant only two reported rapes had been classed as "no crime" so far this year.
Director of Public Prosecutions, Alison Saunders, said: "We have seen an increase in the last few months of referrals coming through, but we need to improve the court process once the victim has decided to engage."
Ms Saunders said she would be in favour of judges warning jurors about the dangers of them having an unconscious bias at the start of a trial rather than at the end.
In a statement, Dame Elish said the investigation and prosecution of rape cases were "uniquely challenging and sensitive", and hoped the review would improve these processes.