Children should be tested for fitness, as well as maths and English, to reduce inactivity, say health campaigners.
Campaign group ukactive says the UK faces a "ticking time bomb" of health problems due to lack of exercise.
Its report says only half of seven-year-olds in England are active for an hour a day, and says more activity should take place in the classroom.
The government says it has increased funding for PE, and that primary schools offer two hours of it per week.
Walking to school
In the "Generation Inactive" report, researchers argue that Body Mass Index (BMI), which measures weight and height, gives little indication of a child's physical fitness and says a child can be slim but still unhealthy.
"We should focus on the health of our hearts, not just the size of our waists," say the authors.
They argue that the way to improve a child's fitness is to incorporate exercise throughout the school day. Walking to school and standing in lessons are ways in which a child's inactivity can be improved.
Former children's commissioner Prof Sir Al Aynsley-Green said: "Whether walking, cycling or being active in and out of PE lessons, providing children with opportunities to be active throughout the day, before, during and after school, is key to engaging even the most disengaged children."
'Kung Fu' punctuation
Teachers at the Montpelier school in Ealing, west London, use "engaging" methods to keep children active in lessons, says the report.
For example, in English lessons, children use "Kung Fu-style hand movements" to explain to the teacher where an exclamation or full stop should go, rather than putting up their hands or shouting out.
The school's head teacher says behaviour at lunchtime has been improved by the increased activity in the classroom.
Lady Grey-Thompson, ukactive's chairwoman and paralympian, said: "The current national ambition focused solely around PE lessons is simply not bold enough. We should aim higher and demand more."
A government spokesman said: "Tackling obesity is a major priority for this government, which is why as part of our plan for education, we want to continue to encourage children to enjoy sports - both in lessons and after school."