Three-quarters of children in England who make a slow start in the "Three Rs" at primary school fail to catch up by the time they leave, data shows.
And more than a third (39%) of pupils who make a bright start are no longer reaching advanced levels when they leave.
The government's school league tables data also shows 9% of primary schools do not meet its floor standards.
Overall 74% of pupils met the required levels in English and maths.
The official data published by the Department for Education shows how more than 14,000 primary schools in England performed on a range measures linked to national curriculum tests, known as Sats.
For the first time the data breaks down school performance for different ability groups and those from different socio-economic backgrounds.
It shows that only a quarter of children, who are classed as having low attainment by age seven, go on to reach the expected level in English and maths, Level 4, by the time they leave primary school.
This compares to the 99% of those classed as high achieving, by the same age, who go on to reach the expected level in both subjects.
However, only 61% of these go on to achieve higher levels of attainment, Level 5, by the time they leave school. This suggests many schools are not realising the full potential of their brighter starters.
The DfE said that:
- in only 27 schools did every low-attaining pupil at age seven go on to reach expected levels in English and maths at age 11
- in 15 schools, a fifth or more of pupils rated high attaining at age seven were not at the expected level in the "Three Rs" by the time they left primary school.
The data also reveals how well pupils from different socio-economic backgrounds progress.
- 80% of pupils on free school meals (FSM) or in local authority care make the required two levels of progress in English
- 85% of non-FSM pupils make this progress
- in maths, 75% of FSM pupils make the required level of progress
- 84% of non-FSM pupils make this progress in maths
Overall, this year schools are doing slightly better than last year with 74% of pupils attaining Level 4 in English and maths compared to 73.5% in 2010.
Fewer than one in 10 schools, or 1,310, are below the government's floor target of achievement - 60% of pupils getting Level 4 in English and maths and scoring above the England median for progress in maths and English.
But 150 have been below the floor for more than five years in a row.
Schools Minister Nick Gibb pledged to pay particular attention to schools which had performed consistently badly over the past five years, whether through conversion to a sponsored academy or other measures.
He added that the data showed how well different schools were educating children of lower ability.
"We need to help schools learn from those head teachers and teachers who deliver a high standard to all those children, including those who struggled at Key Stage 1, or who are from a poorer background.
"We are also shining a light on those schools where pupils who showed early promise did not maintain their bright starts and tailed off to become below average average performers."
Shadow Education Secretary Stephen Twigg said: "The fact that only a quarter of low attainers at age seven go on to meet the expected Level 4 in English of maths when they leave primary school is not good enough.
"The government must use the Pupil Premium as a lever to improve literacy and numeracy at primary schools, by issuing guidance on how best to use the resources, and measuring the impact of the funding."
The data also shows that in 570 or 4% of schools, all pupils reach the expected levels in English and maths.
In total, 11 schools are being investigated for possible maladministration of the tests.
The tables show the best performing local authority is Richmond upon Thames in south London, where 87% of pupils achieve the government's benchmark of Level 4 in maths and English. It is closely followed by Trafford, where 84% achieve this.
The worst performing local authority is Central Bedfordshire, where 66% of 11-year-olds reach this benchmark.
The St Margaret's Church of England School in Halstead, Essex, tops the BBC list of best performing schools, with 100% of pupils achieving Level 4 in maths and English and a full average point score (the average number of points per pupil in the tests).
The school, which only had six Year 6 pupils sitting the tests, also tops the table for higher achieving pupils, with all of them achieving Level 5 (the level expected for 14-year-olds) in maths and English.
It is the only school with 100% of pupils reaching this Level 5 bar.
Crays Hill Primary School in Billericay, Essex, was the worst performing school with just 7% of pupils achieving Level 4 in English and none reaching Level 4 in maths.
This school, which had 15 pupils in Year 6 at the time of the tests, serves the travellers' site at Dale Farm.
In October, the six-acre site in Crays Hill was cleared following a decade-long row over unauthorised plots.