The government is doing "everything we can" to make sure rapist John Worboys stays in prison, Conservative party chairman Brandon Lewis has said.
The Parole Board has said ex-black cab driver Worboys, jailed in 2009 for assaults on 12 women, can be released.
But the government is considering challenging that decision in court.
Mr Lewis said it was crucial victims were "properly protected", while a lawyer for victims said releasing Worboys would "put many women at risk".
Justice Secretary David Gauke is looking into the possibility of applying for judicial review, but will only proceed if there is a good chance of success.
If a judge finds the Parole Board did not follow the correct process or reached its decision on the wrong basis, that decision would be quashed and Worboys would have to make a fresh application for parole.
The Parole Board said it was "confident correct procedures were followed" in the case.
Mr Lewis, who was appointed in the prime minister's recent reshuffle, told BBC One's Andrew Marr Show he understood the public outrage at the decision to release Worboys.
"Every victim out there, every friend and family of victims, everybody who has read about this case will want to know that we are doing everything we can to make sure that the victims are properly protected," he said.
He added: "The Secretary of State for Justice will be doing everything he can to make sure this man stays behind bars."
'Finally being recognised'
Victims' groups and representatives were outraged by news of Worboys's imminent release earlier this month, saying many victims were not informed before it was made public.
Lawyer Richard Scorer, from the firm Slater and Gordon which represented some of the victims, said his clients' "fears and concerns are finally being recognised".
He said the victims hope the government will be able to take action on the parole decision, adding: "We believe this manipulative, dangerous serial sex offender is still a danger to the public and releasing him now would put many women at risk."
Sarah Green, from the End Violence Against Women Coalition, said the justice secretary's efforts were "very welcome".
"Women have been singularly failed throughout this case, from initial investigations all the way through to the parole board decision," she said.
She added that swift action was needed to review the risk Worboys poses to women's safety and to "restore public confidence in the system's ability to do justice and to protect".
At his trial, Croydon Crown Court heard 60-year-old Worboys gave his lone victims drug-laced champagne, saying he was celebrating a big lottery or casino win, before attacking them in his London taxi.
Analysis: A scoping exercise
By Alex Forsyth, BBC political correspondent
There was a huge backlash to the Parole Board's decision to release John Worboys - from those who thought it was the wrong decision and from those who were critical about the way victims had been informed about it.
At this stage, David Gauke's actions are just a scoping exercise, he won't proceed unless he feels there are grounds to do so.
Nonetheless, this is significant. It is highly unusual for a justice secretary to intervene in the decisions of the Parole Board in this way, and that's because the Parole Board is very deliberately independent of government.
It's understood that Mr Gauke, who is new to the job of justice secretary, takes that independence very seriously and wants to maintain it, which is why at this stage he is just collecting information about the possibility of judicial review.
A Ministry of Justice spokesperson said: "Mr Gauke commissioned, mid-last week, advice on the plausibility of a judicial review and the prospect of success of any judicial review.
"The secretary of state is minded to move forward only if there was a reasonable prospect of success."
It came as The Sunday Times reported four cabinet ministers had warned Worboys's release might be unlawful because victims were not consulted.
Must not 'set foot in London'
The handling of Worboys's release has already triggered a government review to look at how the Parole Board reaches its decisions.
Labour's London Mayor Sadiq Khan said in a tweet that he was "astonished" by its decision, saying "it must listen to Worboys's victims".
"He should not be allowed to set foot in London," he added.
Parole Board chairman Nick Hardwick has said hearing the decision must have been "horrible" for the women, but the board was "confident" Worboys would not reoffend.
He said the fact some victims were not informed was a fault with the parole system, but the decision itself would have been carefully considered.