The Evening Standard has apologised after it published details of the Budget via Twitter before George Osborne had delivered his statement.
The newspaper's front page outlined announcements such as the cancellation of a fuel duty rise and a rise in the personal tax allowance to £10,000.
Budget decisions are supposed to remain secret until the chancellor speaks.
The newspaper apologised for what it called a "very serious mistake" and removed the image from Twitter.
Evening Standard editor Sarah Sands said she was "devastated" by the error, in which an image of the newspaper's front page was published via a tweet on the micro-blogging site.
The chancellor has since asked the permanent secretary to conduct a review of the practice of releasing embargoed Budget information in advance of the speech.
He asked that the review report on the appropriateness of the arrangement, which has been in place for many years.
Speaking to the BBC, Ms Sands said "It is an arrangement we have had through successive governments and has always worked. We publish after the Budget.
"This was not online and we had not imagined that it would be tweeted by a very young and inexperienced journalist. I am writing to the Speaker and chancellor now with deep apologies."
She said an investigation was under way and the individual who tweeted the page had been suspended while this takes place.
BBC political correspondent Tim Reid said Treasury sources had played down the leak, saying it was not the first time the Standard had published information about a Budget.
However, sources said the department would be "looking at what had happened", our correspondent added.
Labour leader Ed Miliband, who was reading a photocopy of the newspaper's front page while Mr Osborne was speaking, demanded an investigation into the leak.
In his response to the Budget, Mr Miliband said Mr Osborne "almost need not have bothered coming" to the Commons "because the whole Budget, including the market-sensitive fiscal forecasts, were in the Standard".
He added: "To be fair to the chancellor of the exchequer, I'm sure he didn't intend the whole of the Budget to be in the Standard before he rose to his feet.
"I hope he will investigate and report back to the House."
The front page of the newspaper gave details of some of the chancellor's major announcements on tax, including his scrapping of the annual alcohol duty escalator and the rise in the personal tax allowance to £10,000 by April 2014.
In 1947 Chancellor Hugh Dalton resigned after leaking details of a Budget to journalists.