A Tory MP has quit her government job after voting for a Labour motion to offer free school meals during holidays until Easter 2021.
Caroline Ansell said vouchers were not a long-term solution - but they helped families struggling with the pandemic.
Footballer Marcus Rashford, who is leading a campaign on child hunger, urged MPs to "unite" and stop being influenced by "political affiliation".
On Wednesday evening, MPs rejected the Labour motion by 322 votes to 261.
Home Office Minister Kit Malthouse insisted the government was helping low-income families through the welfare system.
He said the government had raised Universal Credit by £20 a week, adjusted housing benefit to help people with their rent and given £63m to councils to help with hardship funding.
He acknowledged the decision on free school meals was "a tough one" and praised Mr Rashford for his campaign to tackle child hunger.
Responding to the vote, Mr Rashford said "child food poverty has the potential to become the greatest pandemic the country has ever faced" and called on MPs to "face this head on".
"I don't have the education of a politician... but I have a social education having lived through this," he said.
"These children matter... and for as long as they don't have a voice, they will have mine."
Five Conservative MPs rebelled against their party by voting with Labour - including Ms Ansell who has now stepped down as parliamentary private secretary at the Department for Environment, Food and Rural Affairs.
Explaining her decision, she said: "In these unprecedented times I am very concerned to be doing all we can to help lower income families and their children who are really struggling due to the impact of the virus."
She said that food vouchers were "not perfect" arguing that it is better to link meals to activities so children "can also benefit from extra-curricular learning and experience".
However. she added that vouchers could help families in her Eastbourne constituency who were struggling as a result of the pandemic.
The government's stance has also been criticised by Brexit Party leader Nigel Farage, who tweeted: "If the government can subsidise Eat Out to Help Out, not being seen to give poor kids lunch in the school holidays looks mean and is wrong."
But Conservative MP for Bassetlaw MP Brendan Clarke-Smith - who says he was a recipient of free school meals when he was a child - opposed the motion.
"Where is the slick PR campaign encouraging absent parents to take some responsibility for their children?", he asked.
"I do not believe in nationalising children, instead we need to get back to the idea of taking responsibility.
"This means less celebrity virtue signalling on Twitter by proxy and more action to tackle the real causes of child poverty."
Tory MP Ben Bradley posted a series of tweets defending his decision to vote against the motion after getting into a row on Twitter on Wednesday over his opposition to the plan.
He said the vote was not "help poor kids, yes or no" but a promise to "roll out a huge expansion of long term state dependency to millions, when a large [percentage] of those on [free school meals] are not impoverished and don't want or need it."
Mr Bradley defended the support the government offers to poorer families and attacked the Labour, saying: "You'd think a truly 'caring' Labour party could recognise the huge difference between the majority of kids on [free school meals] who are not wealthy by any stretch, but who have good parents and are managing, and impoverished kids who are desperate."
However, the former deputy leader of the Labour Party, Tom Watson, said he could see a U-turn from the government on its decision to go against the campaign, due to wide support in the country.
He told BBC Two's Politics Live: "They don't want a recurring commitment for free school means 52 weeks a year, but we are in a national crisis and there are kids going hungry."
Mr Watson said it was "one of the saddest days of my life" when he listened to a child in his old constituency describe the "feeling in their stomach" when they were going hungry.
"It is a tricky one for the government," he added, "[but] I have got a feeling that if the pressure continues they may have to concede this one.
Children of all ages living in households on income-related benefits may be eligible for free school meals.
In England, about 1.3 million children claimed for free school meals in 2019 - about 15% of state-educated pupils.
Analysis by the Food Foundation estimates a further 900,000 children in England may have sought free school meals since the start of the pandemic.
In Scotland, the government has made £10m available to local councils to continue to fund free school meals over the Christmas, February and Easter breaks. Local authorities that offered provision over the October school break can apply to be reimbursed.
The Welsh government has also pledged to extend free school meal provision to every school holiday until Easter 2021, spending £11m on doing so.
In England and Northern Ireland, however, the scheme will only run during term time.