A "competitive" education system and an "elitist curriculum" do not help address the underachievement of white working class pupils, MPs have heard.
Conversations about issues like "white privilege" and "toxic masculinity" could further alienate them.
Academics told MPs the white working classes faced a "status deficit" as the national conversation had become "much more consumed" with other groups.
They said these pupils were "stuck" in terms of their achievement at GCSE.
The Education Select Committee was taking evidence on Tuesday about the issue of white pupils from disadvantaged backgrounds being left behind.
Matthew Goodwin, Professor of Politics and International Relations at the University of Kent told the committee terms such as "toxic masculinity" and "white privilege" could "become more of a problem as we send yet another signal to these communities that they are the problem".
Asked why white boys and girls in poorer communities performed worse than peers from different ethnic backgrounds, Prof Goodwin said a number of "cultural" factors were likely to play a part.
"What is it that's happening outside of the school environment, that's happening perhaps not only within the family but within society, that is sending these kids the message that higher education or pursuing further education is not for them?
"And that, for example, I would argue, speaking quite candidly, is reflected in our national conversation, which gives these kids what I would call a status deficit - that over the last 10 years our national conversation has become much more consumed with other groups in society.
"That has led in particular the white working class to feel as though they are not being given as much recognition and esteem as others.
"Now that doesn't explain this entire problem, but I think at a macro national level it's absolutely part of the explanation for why even the parents of these kids and their grandparents are feeling as though they are not being given sufficient levels of status and respect in wider society, and lastly in a society that really only gives status to university graduates."
'An unruly group'
Professor Dianne Reay, Emeritus Professor of Education, University of Cambridge, said it was important to look at the history of the education system and ask whether the intention was "to educate and empower the working class children".
"I think if we look at the history we can see it was mainly about curtailing and controlling what was seen at the time as an unruly and disreputable group - and I think that that history dogs us still in the present.
"I think that we currently have a hyper competitive, individualised education system which doesn't play well into the strengths of white working class children and I think we need to look really at what we're offering them.
"A narrow, elitist, exclusive curriculum does not work well, as I say, in enabling working class children to succeed through the system."
Prof Reay said that to understand why other ethnic groups tended to do better, even if they were economically disadvantaged, it was important to understand their "history of migration".
"I think some ethnic groups have come from countries where their families - and they have generations of educational success in their countries of origin and they have social and cultural capital - even though they may be economically impoverished by the move here."
These families were "desperate to make a new start and enable their children to succeed in a new system", she said.
Lee Elliot Major, Professor of Social Mobility at the University of Exeter told MPs that poverty and disadvantage were key issues in underachievement.
"What particularly is an issue for some of these white working class communities is that they live in places where there aren't many opportunities.
"I think we're still suffering from the demise of the manufacturing industry post-war and we haven't actually had a proper plan, whether it's in terms of addressing job opportunities in those areas of the country or about the curriculum and the offering in the school place - I still think these are issues that go back decades.
"You're having here multi-generational issues for these families where parents, grandparents will have had poor experiences in education as well as the children and I think that has profound impacts on attitudes towards schooling."
MPs heard how the educational achievement of these pupils continues to stagnate.
"Around a third of white pupils on free school meals (FSM) get the basic benchmark of five GCSEs with English and maths," said Prof Elliot Major.
"From what I can tell looking at the data, that has stayed pretty much the same for a number of years now, so my reading generally of the data is that they're almost stuck at that kind of level and we've seen other groups progress and we've looked at that and that's been great.
But he said looking at a select committee report from 2014 on the subject made "depressing reading" because there has been "no gain since then".