Toby Young has resigned from the board of a new university regulator after criticism over controversial comments.
Writing for the Spectator, Mr Young said his appointment had "become a distraction" from the "vital work" of the Office for Students.
The right-wing journalist unreservedly apologised for past comments he said were "ill-judged or just plain wrong".
The OfS chairman said Mr Young was "right" to quit and his remarks were "not in line" with the body's values.
Sir Michael Barber said he had done some "good work" in education and to promote greater social mobility.
But he added: "Many of his previous tweets and articles were offensive... he was correct to say that his continuation in the role would have distracted from our important work."
The Department for Education said it remains confident the OfS, the regulator for England's higher education sector, will deliver for students.
'We made a mistake'
A petition calling for Mr Young's dismissal had gathered almost 220,000 signatures.
He faced a backlash after his appointment was announced, with critics attacking his suitability for the role.
The columnist co-founded the West London Free School - which opened in 2011 - and runs the New Schools Network.
The charity promotes free schools in England - of which there are currently 475.
Following his appointment to the OfS, Mr Young deleted up to 40,000 tweets posted since 2009, including references to the size of women's breasts.
Comments made about working class students, about "inclusivity" in a Spectator column in 2012, and remarks aimed at gay people, were also criticised.
Who is Toby Young?
Toby Young started out as a journalist founding the culture magazine Modern Review with Julie Burchill.
Following the collapse of the magazine he was recruited to work on Vanity Fair in New York, an experience that inspired his book "How to Lose Friends and Alienate People".
As a Spectator columnist, Mr Young made a number of controversial comments including a description of wheelchair ramps as part of "ghastly" inclusivity in schools.
In 2011 he set up the West London Free School which placed importance on discipline, ambition and "a competitive atmosphere".
After resigning from his position as CEO of the West London Free School Trust, he said he hadn't "grasped how difficult it is to do better" and that he regretted his earlier criticism of teachers.
His experience with schools led to his appointment to the Office for Students regulator - a move critics argued he was neither suitable nor had the right expertise for.
Chair of the Commons education committee Robert Halfon said remarks Mr Young had made on disability, eugenics and working people went too far.
"If we are to stand up as the Conservative party for what is right," he told BBC Radio 4's Today programme, "we also have to accept when we have made a mistake".
Sir Anthony Seldon, vice-chancellor of Buckingham University, said it was "utterly unacceptable" to have such a person on a public body, involved with education.
In his resignation article, Mr Young said "the caricature" drawn of him during the past week was "unrecognisable".
He wrote: "I am a passionate supporter of inclusion and helping the most disadvantaged, as I hope my track record of setting up and supporting new schools demonstrates.
"But some of the things I said before I got involved in education, when I was a journalistic provocateur, were either ill-judged or just plain wrong - and I unreservedly apologise."
Mr Young thanked Prime Minister Theresa May and the former education secretary Justine Greening for their support.
Mrs May has said she was "not at all impressed" by his previous remarks.
An 'unwelcome distraction' for May
By BBC political correspondent, Alex Forsyth
Toby Young wasn't appointed directly by the prime minister, but the pressure was on her to sack him when the details of his previous remarks emerged.
Her decision to defend his position - while criticising his past comments - allowed opponents to once again question her judgement and authority.
Mrs May's supporters have pointed to Mr Young's achievements in the field of education and said his resignation was a matter for him.
It's unlikely it will have huge or lasting consequences for the prime minister, but just two days into Westminster's new year it's an unwelcome distraction from the authoritative image Mrs May wants to portray, and it adds to the overall perception of a government where things don't tend to run too smoothly.
The Office for Students has been established to hold universities to account on issues like vice chancellors' pay and free speech on campus.
It has powers to fine universities which fail to meet the required standards.
Toby Young's track record setting up & supporting free schools speaks for itself. His decision to stand down from the OfS board and repeat unreserved apologies for inappropriate past remarks reflects his character better than the one-sided caricature from his armchair critics.— Jo Johnson (@JoJohnsonUK) January 9, 2018
The Toby Young saga has cast great doubt on the judgment of the PM who failed to sack him in the first place. Then yesterday we had the spectacle of government universities minister defending his appointment in parliament, he had to go. Tory cronyism could not save his job...— Angela Rayner (@AngelaRayner) January 9, 2018
Critics have welcomed the resignation. Angela Rayner, Labour's education spokeswoman, said events "cast great doubt" on Mrs May, who, she said, had failed to sack him in the first place.
University and College Union general secretary Sally Hunt said Mr Young was right to resign and the "whole sorry episode poses serious questions about the appointments to the board of the Office for Students".
Liberal Democrat leader, Vince Cable, called the appointment "a serious mistake".
But others have scorned the furore.
Piers Morgan said Mr Young had been forced out by the "howling Twitter mob".
"I don't defend his more offensive tweets or columns," the Good Morning Britain presenter wrote, "but I do think he would have brought a valuable and distinctive perspective to university education".
Universities minister Jo Johnson said Mr Young's resignation and apology "reflects his character".