Government borrowing fell by £20bn to £52bn in the year to the end of March, according to official data.
That was the lowest level since the financial crisis of 2008, the Office for National Statistics (ONS) said.
In Chancellor Philip Hammond's Budget last month, the Office for Budget Responsibility had forecast the deficit would be slightly lower at £51.7bn.
The OBR also predicted that government borrowing would rise again this year as tax receipts fall.
The chancellor has said he wants to cut the deficit more slowly than his predecessor George Osborne.
Government borrowing was 2.6% of gross domestic product in the latest financial year, in line with the OBR's forecast.
The hole in the UK's public finances has shrunk since hitting a peak of nearly 10% of GDP shortly after the global financial crisis.
However, economists said that the reduction in borrowing last year was helped by one-off factors.
John Hawksworth, an economist at PwC, said: "It is good news that the deficit is coming down, but it is too soon to be complacent about the state of the public finances.
"As the OBR said last month, a number of one-off factors relating to the timing of tax receipts and spending flattered the deficit figures for 2016-17 but are likely to be reversed in 2017-18."
Higher inflation, an ageing population and rising healthcare costs will continue to put pressure on the public finances, he said.
"So while the deficit is now approaching a more sustainable level, there will still be some tough choices ahead on tax and spending for the next government," he said.
Samuel Tombs, UK economist at Pantheon Macroeconomics, was of a similar view, saying taxes would probably have to rise to help keep down the deficit.
"The OBR expects borrowing to rise to £58.3bn this year, nearly 3% of GDP, as self-assessment tax receipts fall back.
"The chance that the Autumn Budget contains net tax rises - like all of the last six post-election Budgets have done - is very high," Mr Tombs said.
Mr Hammond hinted last week the Conservative party may drop its 2015 pledge not to raise income tax, national insurance or VAT.