Conservative MPs have criticised a campaign for free school meals to be offered over the holidays.
Last week England footballer Marcus Rashford launched a petition urging government to make the change.
Some Tory MPs criticised the campaign, with Brendan Clarke-Smith calling for less "celebrity virtue signalling on Twitter".
But five Tory MPs rebelled against the government to support extending free school meals over the holidays.
Mr Rashford has argued that the number of children with little access to food had grown due to families losing income amid Covid-19 restriction measures.
MPs voted to reject Labour's motion - which called for free school meals to be offered over the school holidays until Easter 2021 - by 322 votes to 261.
Following the vote, Mr Rashford issued a statement that said: "A significant number of children are going to bed tonight not only hungry but feeling like they do not matter because of comments that have been made today.
"We must stop stigmatising, judging and pointing fingers - our views are being clouded by political affiliation."
Earlier on Wednesday, Prime Minister Boris Johnson said he would not change his policy on free school meals, arguing that poor families were supported by the benefits system.
At Prime Minister's Questions, Mr Johnson told MPs: "We support kids of low incomes in school and we will continue to do so.
But he added that the government would "continue to use the benefit system and all the systems of income support to support young people and children throughout the holidays as well."
Earlier this year, a campaign by Mr Rashford pushed the government into a dramatic U-turn when it agreed to extend free school meals over the summer holidays.
Free School meals: what's changed since the summer?
The political debate on how to deal with coronavirus has moved a lot since the summer and we're seeing a different tone from the government.
It's now clear that the virus isn't going away for some time yet and after a summer of big spending, the government seems less willing to put money into one-off temporary solutions.
That feels like part of a broader shift by the Conservatives to try to move away from a fire-fighting "government-by-bailout" approach to something more sustainable.
But there is a big risk in applying that strategy to this issue in particular.
Firstly, nobody wants to see children going hungry over half term or Christmas, so the government needs to be confident they won't slip through the net.
Secondly, Marcus Rashford's high profile campaign means a lot of people are watching.
Compared with some other coronavirus spending, extending free school meals wouldn't cost that much.
So the risk for the government is of a public backlash or as one Conservative MP put it to me, a bit more bluntly, "another self-inflicted and entirely predictable wound".
Responding to the defeat, Labour's shadow education secretary Kate Green said: "Boris Johnson and the Conservatives have badly let down more than one million children and their families.
"No child should go hungry over the holidays, but the government is blocking the action needed to prevent this.
"We pay tribute to Marcus Rashford and others for shining a spotlight on this incredibly important issue. This campaign is not over and the government must reconsider."
Labour point to a "double whammy" of challenges as the furlough job support scheme comes to an end and coronavirus restrictions increase in areas which already have high levels of poverty.
And the party claims nearly 900,000 children in such Covid hotspots will go hungry, unless the government extends a food scheme.
Chief executive of the Child Poverty Action Group Alison Garnham said: "We've reached a low point if in the midst of a pandemic we decide we can't make sure children in the lowest income families have a nutritious meal in the middle of the day."
Some Conservative MPs also criticised the government's approach.
Speaking on BBC Radio 4's Today programme, Harlow MP Robert Halfon said: "I'm not arguing it should go on forever, but the free school meals should at least go on at least until we are out of the coronavirus [pandemic], we hope, God willing, by next spring."
Mr Halfon added that there was significant support for extending the scheme among his fellow Conservative MPs.
And former Conservative former minister, Paul Maynard, said he was "very deeply disappointed" by the government's response.
Five Conservative MPs - Caroline Ansell, Mr Halfon, Jason McCartney, Anne Marie Morris and Holly Mumby-Croft - voted to support Labour's proposal.
But other Conservative MPs criticised the Labour motion.
Bassetlaw MP Mr Clarke-Smith asked: "Where is the slick PR campaign encouraging absent parents to take some responsibility for their children?
"I do not believe in nationalising children, instead we need to get back to the idea of taking responsibility.
"This means less celebrity virtue signalling on Twitter by proxy and more action to tackle the real causes of child poverty."
David Simmonds, MP for Ruislip, Northwood and Pinner, said: "What does it say about the opposition's priorities that all of their interests are simply swept aside in favour of currying favour with wealth and power and celebrity status, spending taxpayers' money to curry favour with celebrity status, wealth and power?
"Now I have no doubt that Mr Rashford is an expert in his own experience, but we should not forget that the experiences he so movingly described took place under a Labour government then supposedly at the peak of its powers in tackling child poverty in this country."
The Welsh government, which recently ordered a three-week lockdown, announced a move to offer food support to struggling families until next spring. Northern Ireland has also extended support for its children to a lesser degree.