Government departments are being asked to identify about £16bn more savings which could be made if needed to pay for new policies or spending demands.
Chief Secretary to the Treasury, Danny Alexander, says they must identify 5% of their budgets as part of measures to tighten financial management.
Departments will also have to monitor and share spending information with the Treasury on a monthly basis.
He said the UK finances would never again be allowed to get into "a mess".
In an interview with the BBC's political editor Nick Robinson, Prime Minister David Cameron said these were "difficult economic times" and if he did not deliver a "nailed down" plan he would not be able to keep interest rates low.
Asked if it meant the public should be braced for more cuts to come, he said: "It always was a multi-year programme, it was never a one-off set of reductions that we had to make - we always set out that plan, the plan hasn't changed, we have to deliver that plan."
But he said he wanted a growing economy and different country that rewarded hard work, reformed welfare and education and was tackling the deficit.
The government is implementing spending cuts in a bid to repair public sector finances, which were hit by the economic downturn following the credit crunch.
Mr Alexander outlined the changes in a speech to the Institute of Fiscal Studies think tank, in which he stressed the importance of "fiscal discipline".
He said the coalition had so far made good progress towards its spending targets for the review period.
As a result the UK had been sheltered from the "worst of the storm that continues to affect our eurozone neighbours".
However, reforms were needed to ensure the plans were not knocked off track by unforeseen costs, he added.
"In an environment of economic uncertainty, with ongoing instability in the eurozone, the UK's large deficit remains a crucial economic vulnerability," Mr Alexander said.
"It remains a clear and present danger to stability."
He said the new rules had been drawn up with finance directors from across Whitehall, "and are designed to fundamentally change and improve financial management across all organisations spending public money".
He said the changes were part of a bid to end a system of financial management in government that has been "stifled by poor information sharing and poor incentives".
"From now on, all departments must monitor and share spending information with the Treasury on a monthly basis. And that data must be consistent."
Whitehall departments are being told to identify 5% of their spending - which works out at £16.3bn of the total £327bn - to act as their department's contingency fund.
"In the spending review, we deliberately kept the reserve small in order to get the most money out to departments," Mr Alexander said.
"It means that departments have to be able to deal with problems that arise from within their own budgets.
"Many departments already operate a small 'unallocated provision' in their annual budgets, to meet smaller pressures that arise.
"And, under the new rules, I have asked all departments to identify around 5% of their resource budget that could be re-prioritised if new pressures emerge or new policies have to be funded, so there is a shared understanding of how it could be paid for."
He said the new controls were "not just a tweak to the Whitehall machine".
"They are another signal of our unwavering determination to deliver the fiscal consolidation we promised," the minister added.
Labour says the government's economic policies are failing the country amid high unemployment figures and sluggish growth and accused the government of delivering a "millionaire's Budget".
Party leader Ed Miliband has repeatedly attacked the government for cutting the top rate of tax from 50p to 45p, for people on more than £150,000 a year - saying the decision had "failed the fairness test".
Mr Cameron defended the decision to lower the top rate of income tax from next April - the government saying the aim was to get more out of the rich by cracking down on tax avoidance measures.
"We are not giving the rich a tax cut, we took the top rate of tax down to make us competitive with the rest of the world but taxes on the rich went up - we have paid for the cost of cutting that top rate of tax five times over from the richest people in the country."