The ablest primary pupils in England almost match those in Taiwan and Hong Kong in maths but fall back by the time they are 16, a study suggests.
The top 10% of English secondary pupils also drop behind those in Australia, Scotland Slovenia and Norway, analysis of international test scores indicates.
But the bottom 10% catch up slightly with their peers overseas according to the Institute of Education research.
Ministers said their reforms would "drive up standards."
The researchers studied results from international comparison tests taken by 10-year-olds in 2003.
Overall English pupils achieved scores in line with the international average for maths in the 2003 Trends in International Mathematics and Science Study (TIMSS), with the top 10% of pupils' results comparable with those from some high achieving East Asian countries.
However analysis of a different set of tests, taken by 16-year-olds six years later, suggested that although the performance of English students overall had not worsened, the brightest had done less well, compared with their counterparts overseas.
The 2009 Programme for International Student Assessment (PISA) tests showed that overall English pupils came 28th for maths out of 65 countries.
The IoE report concludes that the highest-achieving English children appeared to make less progress, relative to their overseas peers, particularly in East Asian countries, between the ages of 10 and 16.
Author Dr John Jerrim described the finding as "worrying".
Last month England's chief inspector of schools, Sir Michael Wilshaw ordered a landmark report into how state schools teach the most able pupils, warning that some pupils got top marks at primary school but did not do as well at secondary school.
He suggested that bright students should be pushed as they would be if they attended independent or grammar schools, adding that too many were left to coast in mixed ability classes or entered too early for exams.
The IoE report recommends more emphasis on ensuring that "high achieving school-children in England manage to keep pace with the highest achieving pupils in secondary school via, for instance gifted and talented schemes".
The report also says that the overall focus should not be on secondary schools because "England is quite clearly some distance behind the leading East Asian nations before children reach their 10th birthday".
Instead the authors recommend that "policymakers should concentrate on educational reforms in primary and pre-school" and want more effort made to raise the basic skills of disadvantaged groups.
They also suggest that in the long run a cultural shift could be needed "towards greater belief in the value of education and the importance of a hard work ethic... to ensure England's future prosperity and economic success in an increasingly competitive world".
Education Minister Elizabeth Truss said: "This report is a damning indictment of Labour's record on education. Based on data from between 2003 and 2009 it shows that our top pupils actually lose ground as they get older, not just with their peers in the Far East, but with those in every country studied.
"This government is clearing up Labour's mess. Our reforms: tougher discipline, more rigorous exams, more freedom for headteachers, a more demanding curriculum and higher quality teaching, will drive up standards so our pupils have a first-class education that matches the best in the world."
But shadow schools minister Kevin Brennan said the report showed that the government had the wrong priorities and should be doing more to provide support in basic skills at primary level as well as strengthening English and maths teaching at secondary level and allowing all students to continue those subjects until 18.
"If we are to be competitive as an economy, we need to give young people the skills and knowledge to succeed in the workplace, not just create a narrow and out of date curriculum and exams, which is what [Education Secretary] Michael Gove is doing," said Mr Brennan.