There is no clear evidence to show that "academies raise standards overall", says a report from the Education Select Committee into England's school system.
MPs are calling for more "openness" in the strengths and weaknesses of such state-funded independent schools which are a majority of secondary schools.
They say that the government should be less "defensive" about academy schools.
Education Secretary Nicky Morgan says academies are "central" to "delivering the best schools".
The report from the cross-party committee of MPs says that standards have risen in the state school system, but it is still too early to determine the impact of academies.
"Current evidence does not prove that academies raise standards overall or for disadvantaged children," says committee chairman Graham Stuart.
"It is clear though that academisation has led to greater competition, challenging many maintained schools to improve and incentivising local authorities to develop speedier and more effective interventions in underperforming schools."
The report calls for much more transparency on what really works in academies.
It argues that these autonomous state schools can be very successful, but the government needs to be much more open about the scheme and to improve supervision.
The report highlights the Ark and Harris academy chains as examples of success.
BBC Education Correspondent Alex Forsyth visited a successful academy in east London
Nestled in a road lined with period properties and flat-fronted shops, The Petchey Academy is a bright modern building full of airy spaces and sharp angles
It was established in 2006 under the last Labour government to replace the struggling Kingsland School.
As a sponsored academy it is backed by the Jack Petchey Foundation and since it opened results have vastly improved.
Last summer 59% of children gained five GCSEs at A* to C including English and maths, compared to 17% at its predecessor school.
The principal, Olivia Cole, said being a sponsored academy has made all the difference.
"I know all academies are different but the model here works extremely well," she said.
"We are in competition (with other schools), but we also talk to each other and support each other through the local authority, and I really think it's helped drive up standards in the whole of Hackney whether maintained schools or academies."
She said the autonomy of academy status has allowed flexibility over the curriculum, while the freedom to set pay and conditions has helped staff recruitment.
"I think it's just a change in attitude. We were directly answerable to the Department for Education, and as a result we knew that if they were going to invest £32m in this amazing building then we had to do something in return, and also for the children this was really a last chance for the children in Hackney."
Pupil Snezana Milivojevic said: "I feel like we have a lot more opportunities than we had in year seven. We have trips to China, Spain, France and Belgium every year. The school has different links, like business partners.
"I'm doing something called careers academy, where I have a business mentor which is helping me find out what kind of career I want to go into which is really good, and not many schools have that so I feel privileged for that."
The MPs also want more clarity over funding, raising questions as to whether the Education Funding Agency can be both the funder and regulator.
"While some chains have clearly raised attainment, others achieve worse outcomes creating huge disparities within the academy sector and compared to other mainstream schools," says Mr Stuart.
There is a call for more transparency in the oversight of academies, including creating a way for schools to leave academy chains and to develop a plan for schools when an academy chain fails.
The report calls for Ofsted to be able to inspect academy chains - and last week the Department for Education conceded that inspectors would be able to carry out such inspections, but without delivering a formal judgement.
While academies have become a majority in secondary school they remain a minority of primary schools.
'Matter of urgency'
Before pressing ahead with an expansion of primary academies, the MPs say, research into their value should be commissioned as a "matter of urgency".
"There is at present no convincing evidence of the impact of academy status on attainment in primary schools," says the report.
"The Department for Education needs to be far more open about the implementation of the academies programme: it has much to gain from transparency," the MPs conclude.
On the quality of free schools, the report says it is "too early to draw conclusions". But it calls for more clarity over funding and better co-ordination with local authorities.
The education secretary said academies and free schools have helped by "promoting new ideas and approaches, and helping to drive up standards in other local schools as a result".
"They have also created greater choice and are more accountable to parents and communities who have a much greater opportunity to hold them to account than has been the case with schools in the past.
"The interests of parents, pupils and communities are at the heart of the programme," said Mrs Morgan.
Tristram Hunt, shadow education secretary, described the report as a "damning verdict".
"The report finds that under David Cameron there is no convincing evidence that schools policy has delivered improvements."
Becky Francis, professor of education at King's College London, said the report illustrates that "the evidence on whether or not academies have had more success in raising attainment than other equivalent schools is mixed, and hard to pin down".
This is in part because there are many different types of school under the "academy" label and progress is from very different starting points.
Kevin Courtney, deputy general secretary of the National Union of Teachers, said this was "an utterly damning report".
"Those parents whose schools have been forcibly converted to academies, often against their wishes and those of the staff, will rightly question just whose interests the government has been pursuing in the last five years."
Mary Bousted, leader of the ATL teachers' union, said the report showed: "Academy status is no magic potion to transform schools."