Eating chips, chocolate and cake may be damaging to a child's intelligence, according to researchers at Bristol University.
Their study suggests a link between a diet high in processed foods and a slightly lower IQ.
Writing in the Journal of Epidemiology and Community Health, they suggest poor nutrition may affect brain development.
The British Dietetic Association said more young parents needed to be educated about healthy eating.
The eating habits of 3,966 children taking part in the The Avon Longitudinal Study of Parents and Children were recorded at the ages of three, four, seven and eight and a half.
The researchers said three types of diet emerged: Processed diets which were high in fat, sugar and convenience foods, traditional diets of meat, potato and vegetables, and health conscious diets of salads, fruit and fish.
The children all took IQ tests when they were eight and half.
The researchers found a link between IQ and diet, even after taking into account other factors such as the mother's level of education, social class and duration of breast feeding.
A diet high in processed food at the age of three was linked to a slightly lower IQ at the age of eight and a half, suggesting early eating habits have a long term impact.
Dr Pauline Emmett, who carried out the study at Bristol University, said: "Brain development is much faster in early life, it's when it does most of its growing. It seems that what happens afterwards is less important."
Although the relationship between diet and IQ was very strong, the impact was quite small. Processed foods were linked with IQs only a few points lower.
Experts in the field said the results had confirmed common sense.
Fiona Ford, spokesperson for the British Dietetic Association, said: "It's well worth looking at the long term impact of diets, everyone's familiar with the short term.
"The research confirms the type of advice we already know, but that's not always enough. Sometimes a society has to help a person change, we need to be educating more young parents about healthy eating."
Kristian Bravin, dietician at Leicester Royal Infirmary, said: "Most people know what they should do, some people don't have the inclination to cook good meals.
"I'm all in favour of a little bit of what you fancy, but when you're doing it every week it's a problem.
"People should seek advice from a registered dietician, but simply it's a message of moderating fat intake, five fruit and veg a day and whole grain starchy foods."