Chancellor Philip Hammond has said he will prioritise spending on new homes and transport rather than following his predecessor George Osborne's aim to balance the books by 2020.
He told the Conservative conference the deficit was still too large and would need to be tackled "in due course".
But he said the Brexit vote may cause "turbulence" and business confidence would be on a "bit of a rollercoaster".
He said that it was "common sense" to invest to support growth and jobs.
- Rolling updates on the Conservative conference
- Pound falls as May indicates Brexit date
- Kuenssberg: Out with the hawks - in with the pragmatists
- How can the UK build more homes post-Brexit?
- Javid attacks 'nimbyism' over new homes
- Fox: Seize 'golden' Brexit opportunity
- Infrastructure - the economy's magic bullet?
In his conference speech, Mr Hammond said his predecessor's deficit reduction policies "were the right ones for that time" but that times had changed since the vote to leave the EU, which he said had caused uncertainty for business.
"When times change, we must change with them," he said. "So we will no longer target a surplus at the end of this Parliament. But make no mistake the task of fiscal consolidation must continue.
"The British people elected us on a promise to restore fiscal discipline. And that is exactly what we are going to do. But we will do it in a pragmatic way that reflects the new circumstances we face."
His speech came against a background of the pound falling to its lowest level against the dollar since early July, after the timetable for starting Brexit negotiations were set out on Sunday.
Addressing the party faithful he said that as part of a new "flexible and pragmatic" plan, the details of which will be fleshed out in next month's Autumn Statement, there would be greater scope for investment to boost the economy, including extra borrowing of £2bn to speed up the construction of new homes.
Mr Hammond said the government would use "all the tools at its disposal" to increase the amount of new housing stock "because making housing more affordable will be a vital part of building a country that works for everyone".
Analysis by Laura Kuenssberg, BBC political editor
Out with the deficit hawks - in with the conservative pragmatists.
That might not, well, quite catch on. But there is certainly a sense of down with the old regime at the Tory conference today. Philip Hammond isn't just a totally different character to the former occupant of the job, George Osborne, but he is intent on junking a fair bit of Mr Osborne's approach too.
His targets for getting rid of the deficit are being ditched. Let's face it, George Osborne missed them time and again in any case, but it is politically a big shift for Mr Hammond and Mrs May to tear up the previous borrowing rules, when balancing the books was the central mission of the government they were all part of.
Instead, there will be a new "framework" but sources close to Mr Hammond believe that setting specific targets is a fool's errand. But given that Mr Osborne's target was extremely unlikely to be met, the political change is perhaps bigger than what it means in practice.
Asked whether the public investment plan and the refusal to set out a timetable for eliminating the deficit marked the end of the "age of austerity", Mr Hammond told the BBC a "credible plan" was still required to get the public finances into the black.
But he told Radio 4's Today: "As we go into a period where inevitably there will be more uncertainty in the economy, we need the space to be able to support the economy through that period.
"If we don't do something, if we don't intervene to counteract that effect, in time it would have an impact on jobs and growth."
Mr Hammond rejected claims there was now little difference between the Conservatives and Labour on economic policy, saying his plans were "pragmatic and measured" while the opposition - which has pledged to borrow to spend £500bn on the UK's infrastructure - were living in "la-la land".
But Tory MP Sir Desmond Swayne, a minister under David Cameron, told the BBC he was "nervous" about the move as he believed the government's goal should be to eradicate the deficit entirely.
"I am concerned but I am ready to listen and I have great confidence in Philip Hammond," he told Radio 4's PM programme.
Labour said it welcomed the government's "u-turn" on borrowing but said ministers were still going ahead with cuts to in-work benefits and local authority grants.
"As well as abandoning their own fiscal charter, this was full of the same empty promises George Osborne made," said shadow chancellor John McDonnell.
The SNP said the government was "incredibly complacent" over weak domestic productivity while the Lib Dems said Mr Hammond has already "lost the most important battle" over continued membership of the EU's internal market.
In business reaction, the CBI said it was right to adopt a more flexible approach to fiscal policy "at this point" while the British Chambers of Commerce called for "unambiguous commitments to runways, roads and railways".
The economy is taking centre stage on the second day of the Tory conference with Mr Hammond also promising £220m to help tech firms realise their ideas in the UK and guaranteeing firms which secure multi-year funding from the EU between now and the UK's exit that the UK will plug the gap after Brexit.
Earlier on Monday, Communities Secretary Sajid Javid set out new measures aimed at building a million new homes by 2020, saying existing house-building levels were totally inadequate.
The government will borrow £2bn to support the "Accelerated Construction" scheme, which aims to get houses built on publicly-owned "brownfield" land available for swift development. There will also be a £3bn Home Building Fund to provide loans to stimulate projects.
Mrs May has said she will set the ball rolling on Brexit by the end of next March by formally triggering Article 50 of the Lisbon Treaty, paving the way for the UK to be outside the EU by the summer of 2019.
In a statement on Monday, the European Commission said it would "work constructively" with the UK once negotiations about its withdrawal began but there would be no talks until the formal process started.
"When it comes to Article 50 we will work constructively on the basis of a notification, not of a speech, and until this letter of notification arrives there will be no negotiation," a spokesman said.