Mr Cameron added: "The idea that wearing a poppy to remember those who have given their lives for our freedom is a political act is absurd.
"Wearing a poppy is an act of huge respect and national pride."
"Such initiatives would open the door to similar initiatives from all over the world, jeopardising the neutrality of football," it said.
Fifa has allowed a period of silence to be held prior to the game against
, which takes place the day before Remembrance Sunday.
During Prime Minister's Questions in the House of Commons, Mr Cameron said a clear message needed to be sent to Fifa from the government.
Asked about the matter, he replied: "I think [the questioner] not only speaks for the whole House, but in fact the whole country, [in] being completely baffled and frankly angry [at] the decision made by Fifa.
"If teams want to be able to put the poppy on their shirt, as many teams in our football league do, they should be able to at the national level, whether it is the English team or whether it is the Welsh team.
"I think this is an appalling decision and I hope they'll reconsider it."
England will wear poppies on their training kit and will stand for a two-minute silence during practice on Friday. The players will also wear poppy-embossed jackets during the national anthems on Saturday.
The Royal British Legion has backed the FA's efforts to mark Remembrance Sunday.
"There are other ways to honour the poppy than by wearing it on a shirt," said its director general Chris Simpkins.
"We are satisfied that England will enter the competition knowing they have shown proper respect for our Armed Forces."
Wales, who play Norway in Cardiff on Saturday, are set to wear a poppy on their training shirts for the warm-up and tracksuits for the national anthem, with the words "Cymru'n Cofio" ("Wales remembers") underneath the poppy.
The Football Association of Wales supported the FA's request to Fifa for permission to wear a poppy on their kit.
Scotland, who face Cyprus in a friendly on Friday, will also wear poppy-emblazoned training tops.
DID YOU KNOW?
No poppies were worn when England played Brazil on 14 November 2009
England Under-21 manager Stuart Pearce insisted that Fifa's ruling would not detract from the England squad's observance of the occasion.
"Whether I understand it or not, you ask the question, they give a decision and you get on with it. That is the nature of it. But whether or not you have anything emblazoned on your shirt, it is what is in your heart that matters," said Pearce.
"We would have liked to have done it as an organisation and a country, but the powers that be say they don't want us to.
"That is the end of it, but it will not diminish what is in my heart and the respect for those who have given their lives for this country."
Former England captain Alan Shearer condemned the decision.
"I think it's terrible, I really do. I think they should let it go ahead and I don't understand their decision at all," he said.
"Everybody seems to agree it is wrong but I don't think Fifa will change their mind.
"I don't think relationships between the FA and Fifa are great so I'd be surprised if they were to change their minds now, but I just think it is a terrible decision."
Former FA chief executive David Davies told BBC Radio 5 live that the policy was necessary but needed tweaking.
"Around the world, if you didn't have this rule, people would use it for political or religious or for personal slogans," he said.
“The British public feel very strongly about this issue - it is not religious or political in any way”
Hugh RobertsonMinister for Sport and the Olympics
"One of the first countries screaming out in protest would be our home nations. [But] things have changed. People do respect it [the poppy].
"After the game the FA must campaign, perhaps with the Germans, against this far too draconian and ill-thought-through policy."
On Tuesday, Sports Minister Hugh Robertson wrote to Fifa seeking permission for the England and Wales teams to wear poppies.
He commented: "Wearing a poppy is a display of national pride, just like wearing your country's football shirt.
"The British public feel very strongly about this issue - it is not religious or political in any way."
British Fifa vice-president Jim Boyce added: "I think there has to be a bit of common sense used when requests like this come in.
“England team should wear poppies on Saturday. It's the nation's tradition and it would be disrespectful not to”
"Armistice Day is a very important day in the FA calendar, as it is with other associations, and I don't think it would offend anybody to have a poppy on the shirts."
Injured England midfielder
Jack Wilshere tweeted
: "My great-grandad fought for this country in WW2 and I'm sure a lot of people's grandparents did.
"England team should wear poppies on Saturday. It's the nation's tradition and it would be disrespectful not to."
England did not wear poppies for games close to Remembrance Day against Argentina on 12 November, 2005 and Sweden on 10 November, 2001.
But an FA spokesman said "a greater focus has been given to the level of support and respect shown by the national teams" over the past five years.
He said: "Since 2005, our clubs have all begun to wear poppies on their match shirts in domestic games for the early part of November as a mark of respect for those who lost their lives serving their country.
"The FA and England team have built very strong relationships with Tickets4Troops, Help for Heroes and the Royal British Legion over the past five years.
"As part of this growing commitment, we wanted to show our respect and support this weekend by wearing the poppy and our players are very passionate and vocal about this."
On Wednesday, Fifa confirmed to the BBC that two protestors from the English Defence League were on the roof of their headquarters in Zurich to protest Fifa's refusal to lift the ban.
England and Wales' rugby league players will be wearing poppies this weekend, along with the other two teams - Australia and New Zealand - involved in the Four Nations.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.