There is no alternative to making "prisons work", David Cameron has said, insisting criminals can be punished and rehabilitated at the same time.
In a speech in London, he said the debate on crime and punishment had become too "black or white".
Serious offenders must be imprisoned, but jails must have a "positive impact" on inmates, he argued.
The PM has had a difficult few days, with Andrew Mitchell quitting as chief whip and confusion over energy policy.
In a long-planned speech to the Centre for Social Justice, Mr Cameron sought to regain the initiative by insisting crime was an issue that "matters to all of us" and rejected characterisations of his views from both the left and right of the political spectrum.
He referred to comments he made while opposition leader in 2005, following which he was accused of wanting to "hug a hoodie".
Mr Cameron said: "For many people, I am associated with those three words, two of which begin with 'h' and one of which is hoodie... even though I never actually said it.
"For others, I am the politician who has argued for tough punishment. So do I take a tough line on crime or a touchy-feely one?
"In no other debate do the issues get polarised like this... with the crime debate, people seem to want it black or white, 'lock 'em up' or 'let 'em out', blame the criminal or blame society, 'be tough' or 'act soft'."
Personal responsibility was at the heart of the criminal justice system, he stressed, meaning long prison sentences were the only "thinkable" punishment for certain serious offenders.
"This is what victims and society deserve... And the society bit matters. Retribution is not a dirty word; it is important to society that revulsion against crime is properly recognised, and acted on by the state on our behalf," he argued.
But echoing comments made by Tony Blair in the 1990s, Mr Cameron said the government must "think hard about dealing with the causes of crime" not just the results of crime.
This, he stressed, meant more emphasis on crime prevention and, at a time when budgets were being cut and prison numbers stretched, priority being given to reducing re-offending.
Critics have warned that Ken Clarke's replacement by Chris Grayling as justice secretary last month signalled a hardening of the approach on sentencing, but the prime minister said he was still as committed to a "rehabilitation revolution" for prisoners.
Private firms and charities must be given an expanded role to work with all prisoners, not just those in prison for a year or more, he said, while the model of payments by results for such firms had to be accelerated.
"I say let's use that time we have got these people inside to have a proper positive impact on them... it is not a case of 'prison works' or 'prison does not work' - we need to make prison work better.
"And once people are on the outside, let's stick with them, and give then proper support."
Mr Grayling told BBC Radio 4's Today programme: "We have to do this differently. We have got people coming back out onto the streets after prison who are as likely to reoffend again as not to reoffend.
"The benefit of a payment-by-results system is it forces the organisations working with you to look for what really does work because they don't get paid unless they do."
Plans were announced on Sunday to introduce a new offence of possessing firearms with an intention to supply them to others, carrying a maximum life sentence, designed to target "middle men" who import and traffic weapons for gangs.
Labour said the coalition had cut police numbers and budgets, circumscribed judges and "let victims down".
"If the government's going to make a serious announcement this week he (David Cameron) should explain why he's done nothing for the last 29 months and he's got to explain how these policies are going to be paid for," said shadow justice secretary Sadiq Khan.
Rhodri Davies, policy manager of the Charities Aid Foundation, said: "The prime minister is right that payment-by-results contracts have potential to help charities use their expertise to tackle intractable social problems such as reoffending.
"But ministers need to improve the way these contracts are designed so charities are not simply squeezed out in favour of large private sector providers."