Pressure for quick fixes can outweigh research evidence when ministers set schools policy, according to a study of three decades of education initiatives.
Media pressure and political expediency are more likely to influence decision making, says a report from the CfBT Education Trust.
The report draws upon interviews with former ministers and civil servants.
It calls for the setting up of an independent chief education officer to give objective advice.
The report, Instinct or Reason, due to be published next week, examines the pressures that have shaped education policy since the late-1970s, across Conservative and Labour administrations.
Using interviews with previous education ministers, academics and education figures, the report highlights how decisions can be taken without adequate research-based evidence.
The figures contributing to the report included former education secretaries Estelle Morris, David Blunkett and Gillian Shephard, senior civil servant Michael Barber and head teachers' leader, John Dunford.
The report found that the short-term interests of the media had a disproportionate influence, making it difficult for a government to develop effective long-term strategies.
Short-lived ministerial careers could also add to the difficulties in developing longer-term planning.
A sense of "something must be done" could rush governments of all parties into taking measures to appear to be in control, researchers were told.
Political pressure could also be a key factor, with political advisers being given greater consideration than education experts.
Researchers identified a trend for prime ministers and their special advisers to be increasingly interventionist, with their views having to be incorporated in policy.
In contrast, the status of education experts and academic researchers was seen to have lowered in recent years.
The report also suggests a pattern of undue attention being paid to international comparisons, even though the overseas examples might not be relevant.
However while political and media pressure are taken seriously, public opinion on education plays a less influential part in decision making, researchers concluded.
The research also found that the longer a government was in power, the less likely it was to be influenced by evidence, trusting in its own instincts instead.
The report looks for ways to close the "gap between evidence and policy making".
It suggests that the education system needs a more effective approach to evaluating ideas, in the way that the National Institute for Health and Clinical Excellence (Nice) can provide independent guidance on health issues.
And it says there needs to be a way of raising the profile of research and independent experts, who might cast a more practical light on the political agenda.
Estelle Morris, in the report's introduction, says that it is important to understand more about how policy is made.
And she says the findings raise questions about why academic research and expert opinion do not play a larger part in informing decisions.
"The relationship between education politics and education research has not been an easy one," says the former education secretary.