David Cameron says it is his "mission" in politics to make the Big Society succeed - amid claims it is being wrecked by spending cuts.
The PM told social entrepreneurs the initiative would get all his "passion" over the five-year Parliament.
The government has also set out details of how a Big Society bank to fund voluntary projects will work.
Labour leader Ed Miliband said Mr Cameron was undermining his own Big Society concept through spending cuts.
Various charity figures have suggested that the scale of local authority cuts could kill off the idea before it gets going - by "destroying" existing voluntary groups.
The Archbishop of York, Dr John Sentamu warned on Sunday that investment in public services was needed if the Big Society was to succeed.
Mr Cameron said the government's "duty" was sorting out the budget deficit and building economic growth but he added: "What is my mission, what is it I am really passionate about? It is actually social recovery as well as economic recovery.
"We do need a social recovery to mend the broken society and to me, that's what the Big Society is all about."
He said he recognised that measures taken to tackle the deficit "will make me unpopular" but said he saw it as his duty, adding: "I don't believe it is impossible to do your duty at the same time as having a sense of mission and purpose about what would make this country stronger, better, a nicer place to live."
Mr Cameron said Britain needed people to take more responsibility and government action could only ever be part of the answer to solving society's problems. He said he wanted to make it easier for people to volunteer.
Addressing criticism that the Big Society idea was too vague, he said it was not just about rolling out one single policy: "What this is all about is giving people more power and control to improve their lives and communities."
And he denied it was a cover for spending cuts, saying he had been promoting the idea since becoming Tory leader in 2005: "It's not a cover for anything, it is a good thing to try and build a bigger and stronger society whatever is happening to public spending."
He said he kept talking about the Big Society "because it is what I care about" adding that was why it was in his manifesto "rather than just because it makes a great headline on the Ten O'Clock News".
In an interview with BBC political editor Nick Robinson, Mr Cameron said government funding cuts to local government was merely returning to the levels of 2007. He added that 75% of charities did not get any money from the government at all.
He said: "I hope that local authorities will be able to work with charities that they fund and work out how best they can handle them."
Asked if he was suggesting people blame their local council for cuts, he said: "No, clearly everyone in the end will blame the government for having to make cuts. I hope they will fairly say that we're doing this not because we want to but because we have to, because of the inheritance we've had from the last Labour government."
And in response to suggestions that people believed he was "Margaret Thatcher all over again", he said: "I am different to Margaret Thatcher, different to past Conservative governments, this whole idea of emphasising the importance of building the Big Society and all the things that we can do - government is not just making cuts, sitting back and saying 'let's hope society steps forward'."
But Labour leader Ed Miliband told the BBC the government had cut local authority budgets by more than one quarter and the fact the council cuts were "front loaded" - heavier in early years - meant they did not have the time to adapt and find savings elsewhere.
"What Mr Cameron doesn't seem to realise is that because he's cutting too far and too fast he is undermining the very idea he claims to champion."
He added: "If he claims to support the Big Society then he should be worried about the closure of local libraries, the threat to children's centres, the closure of Citizens Advice Bureau services, because all of those things are a crucial part of the Big Society and if he carries on as he is they are going to shut down and he is going to undermine the very idea he claims is his mission in politics."
And at the event on Monday Sir Stephen Bubb, head of the Association of Chief Executives of Voluntary Organisations, said he supported the idea of the Big Society but was worried about cuts to voluntary organisations.
He told the PM: "Much of this is coming from local councils. I think you need to think about how you say to local councils the cuts they are making are disproportionate, they are hurting disadvantaged communities."
TUC general secretary Brendan Barber said the Big Society showed that Mr Cameron "believes that policies of slash, burn and sack will make all our lives better".
The government has also announced details of a Big Society bank, which will use £100m in its first year from dormant bank accounts and £200m from the Project Merlin deal with banks to support working capital projects approved by the government. Among its advisers will be the Labour donor Sir Ronald Cohen.
BBC business editor Robert Peston said the bank would act as a wholesaler - it will not put money directly into social enterprises or businesses with a social purpose but would invest in the funds and operations which make direct investments.