Iain Duncan Smith has resigned as work and pensions secretary, denouncing £4bn of planned cuts to disability benefits as "indefensible".
He complains of pressure to "salami slice" welfare, saying the latest cuts were a "compromise too far" in a Budget that benefits higher earning taxpayers.
David Cameron said he was "puzzled and disappointed" at the resignation.
Earlier, the government had indicated it would look again at some of the proposed disability benefits changes.
BBC political editor Laura Kuenssberg said: "There had been bad blood off and on between Chancellor George Osborne and Iain Duncan Smith over some of the more controversial welfare reforms, but nobody expected this move only 48 hours since the Budget."
She added that she understood Mr Cameron had personally tried to persuade Mr Duncan Smith to stay on and called the resignation "a bombshell at a very sensitive time".
In his Budget on Wednesday, Mr Osborne had said the government would be spending an extra £1bn on disability but changes to disability benefits announced a few days earlier had suggested the government would save £4.4bn by 2020-21.
They included changes to Personal Independence Payments (PIP), which will replace Disability Living Allowance (DLA) in January 2017, that were expected to save £1.3bn and sparked outcry from opposition parties and some Tory MPs. A government source said on Friday they would be "kicked into the long grass".
Mr Duncan Smith, who was the Conservative Party leader and Leader of the Opposition from 2001 to 2003, wrote in his resignation letter that the changes to disability benefits were "defensible in narrow terms, given the continuing deficit".
But he said they should have formed part of "a wider process" of finding the best way to focus resources on those most in need.
"I am unable to watch passively whilst certain policies are enacted in order to meet the fiscal self-imposed restraints that I believe are more and more perceived as distinctly political rather than in the national economic interest," Mr Duncan Smith said.
"Too often my team and I have been pressured in the immediate run up to a Budget or fiscal event to deliver yet more reductions to the working-age benefit bill.
"There has been too much emphasis on money-saving exercises and not enough awareness from the Treasury, in particular, that the government's vision of a new welfare-to-work system could not be repeatedly salami-sliced.
"It is therefore with enormous regret that I have decided to resign."
He wrote that he believed the cuts "would have been even fairer to younger families and people of working age if we had been willing to reduce some of the benefits given to better-off pensioners".
In his reply to Mr Duncan Smith, Mr Cameron said they had all agreed that "the increased resources being spent on disabled people should be properly managed and focused on those who need it most".
"That is why we collectively agreed - you, No 10 and the Treasury - proposals which you and your department then announced a week ago," he said.
"Today we agreed not to proceed with the policies in their current form and instead to work together to get these policies right over the coming months.
"In the light of this, I am puzzled and disappointed that you have chosen to resign."
By Laura Kuenssberg, BBC political editor
A source close to Iain Duncan Smith told me his resignation is "because of their behaviour. This is not about Europe"
Iain Duncan Smith is understood to have told them that the government "can't keep taking money from the working poor."
He was concerned that his "narrative for welfare reforms was in danger of being thrown away." He felt he had no choice but to act because welfare reforms "matter more than anything".
Over the weekend Iain Duncan Smith discovered the Chancellor planned to offer cuts in Capital Gains Tax and was very unhappy that those tax cuts were to be offered to the better off, while he had been forced to make more welfare cuts prematurely, in his view. When Number 10 and the Treasury then backtracked on the reforms to PIP today, he concluded that he could no longer remain in government.
Sources close to him are absolutely adamant that his decision was in no way related to his views on Europe.
Mr Duncan Smith was praised by Tory MP and fellow Out campaigner in the EU referendum Peter Bone as a "man of principle" who had taken the job because he "passionately believed in social justice".
And Tory Eurosceptic MP Jacob Rees-Mogg said that Mr Duncan Smith had been "as important a welfare secretary as I can think of".
"I think it is deeply concerning that a man of the stature and standing of Iain Duncan Smith should be pushed into this position," he added
However, opposition parties welcomed Mr Duncan Smith's resignation and called for Mr Osborne to resign as well.
Labour leader Jeremy Corbyn said: "The resignation of Iain Duncan Smith reveals a government in disarray and a chancellor who has lost the credibility to manage the economy in the interests of the majority of our people.
"The chancellor has failed the British people. He should follow the honourable course taken by Iain Duncan Smith and resign."
Scotland's First Minister Nicola Sturgeon said on Twitter: "IDS resignation may be more about EU than his conscience - but he has struck a fatal blow to #PIP cuts and that's a good thing."
Liberal Democrat leader Tim Farron said: "It is quite right that Iain Duncan Smith has resigned over this heartless plan to cut disability benefits, but the true mastermind of the changes, George Osborne, should also be considering his position."