The chief civil servant at the Department for Education has been sacked following the row over A-level and GCSE results in England.
Jonathan Slater was due to stand down next year, but will now leave the department by next week.
A government statement said Boris Johnson "concluded that there is a need for fresh official leadership".
But the civil service union accused No 10 of "discarding" its members to "keep scrutiny from the government's door".
Mr Slater is the fifth permanent secretary to leave his post in six months.
The news comes a day after the head of exam regulator Ofqual, Sally Collier, also resigned from her role.
Thousands of A-level students saw their results downgraded earlier this month due to an algorithm designed to moderate them.
It led to a huge backlash and a u-turn by government ahead of the publication of GCSE results, reverting the grades back to those awarded by teachers.
Education Secretary Gavin Williamson faced calls to resign, but No 10 said it had full confidence in him.
Mr Slater has been the permanent secretary at the DfE for four years and was due to step down in Spring 2021.
He will now be replaced by Susan Acland-Hood, who was brought into the department on a temporary contract last week to lead on its exam response.
Cabinet Secretary Sir Mark Sedwill thanked Mr Slater for his 35 years as a public servant and the government said a permanent replacement would be confirmed in the coming weeks.
Mr Williamson also thanked Mr Slater for his "commitment to public service", adding: "Like the prime minister, I appreciate the hard work of officials across government, particularly during this unprecedented time."
The education secretary said he and his new permanent secretary's "immediate focus remains on making sure every child returns to the classroom full-time at the start of term".
The general secretary of the FDA union, Dave Penman, criticised the decision to sack Mr Slater.
He said: "If it wasn't clear before, then it certainly is now - this administration will throw civil service leaders under bus without a moment's hesitation to shield ministers from any kind of accountability."
He accused the government of "scapegoating" civil servants and claimed trust between ministers and civil servants was "at an all-time low".
The Labour Party also condemned the move, saying civil servants had "time and time again taken the fall for the incompetence and failures of ministers".
Shadow education secretary Kate Green said: "Parents will be looking on in dismay at a government in complete chaos just a matter of days before children will return to schools.
"Leadership requires a sense of responsibility and a willingness to be held accountable, qualities this prime minister and his ministers utterly lack."
Former head of the Home Civil Service, Sir Bob Kerslake, who has worked as a Labour advisor, called the sacking "a disgrace", telling Times Radio that senior civil servants were "carrying the can for the failure of ministers".
So, what does the departure of Jonathan Slater mean - and why does it matter?
For his union, the FDA - and for Labour - it is straightforwardly a sign that, when things go wrong, the buck now firmly stops with the officials and not government ministers.
Angry Conservative MPs were being privately reassured that "heads would roll" after the exams controversy.
And now, both a senior civil servant and the head of Ofqual have now departed, while Gavin Williamson and his education ministers remain in post.
But something of a pattern is now emerging.
In February, the most senior official at the Home Office resigned - and took the government to court, claiming there had been a "vicious and orchestrated campaign" against him.
Other senior civil servants have made less of a fuss, but have nonetheless left their jobs.
The most senior Whitehall mandarin - Sir Mark Sedwill - recently moved, the head of the Foreign Office announced an earlier than expected departure, and it was confirmed last month that the permanent secretary at the Ministry of Justice would be leaving too.
So, not-so-permanent secretaries seems to be a feature of this administration.
Cabinet Office Minister Michael Gove has talked about reforming the civil service. In a speech in June, he said government departments recruited in their own image and their assumptions were "inescapably metropolitan".
So a strategic rethink and an increased turnover of senior Whitehall personnel are probably not entirely unrelated.
But what might worry senior civil servants more is they might be sacrificed for short-term news management, rather than as the result of any strategic master plan.
And there is a risk that this, in turn, might affect the quality of those who apply for senior civil service roles in the future.