Sales on the year's biggest shopping weekend dipped for the first time in seven years, according to the National Retail Federation.
US consumers spent around $1.7bn less over the holiday weekend, with the average shopper spending $407.02 from Thursday to Sunday.
That's down from $423.55 in 2012.
Retailers blamed stagnant wages and economic uncertainty for keeping wallets shut, as they slashed prices to lure reluctant shoppers.
In total, the National Retail Federation estimates that US shoppers spent around $57.4bn this year, down 2.7% from $59.1bn last year.
Sales on Black Friday itself were down, as retailers opened stores on Thanksgiving Thursday and offered more details earlier in the week to entice shoppers.
According to ShopperTrack, a market research firm, Black Friday foot traffic was down 11% and sales slumped by 13%.
Cyber Monday solution?
Despite the gloomy start to holiday shopping season, the National Retail Federation said it still expected holiday sales to surpass last year's spending.
One beacon of hope for nervous retailers is that US consumers have spent more on online shopping than ever before.
"We expect Cyber Monday to be bigger than ever," said National Retail Federation's Matthew Shay in a statement.
The day, launched in 2005 by online retail association Shop.org, became the biggest US online shopping day in 2011, according to comScore.
But it's not just one day any more: in an effort to lure penny-pinched shoppers, Wal-Mart, the world's largest retailer, declared this entire week "Cyber Week" and said it would offer sales every hour throughout the period.
However, Wal-Mart, Target, and Best Buy have all lowered their holiday shopping forecasts, pointing to declining consumer confidence, high unemployment, and increasing price competition as hurting their bottom lines.
But if US consumers are unwilling to part with their dollars, international brands may take comfort in the increasingly global nature of Black Friday.
According to Borderfree, a company that helps build global online shopping sites for companies like Macy's and J. Crew, US retailers saw online sales to international consumers rise 50% this Black Friday compared to last year.
Shoppers in Russia, South Korea, China, Germany and Sweden were the biggest spenders.
And Borderfree says retailers can expect a "second peak" on Cyber Monday.
By one estimate, shoppers on the UK are expected to shell out £450m, or 16% more this year