Nearly two-thirds of people asked in the UK believe taxpayers have paid too much to cover the cost of London 2012.
However, 55% of those questioned in a poll commissioned by BBC Radio 5 live said the Olympics would prove good value in terms of benefits to the UK.
The results were revealed as the 100-day countdown to the opening ceremony of the Games began.
A total of 2,007 adults were surveyed earlier this month for the poll carried out by market research firm ComRes.
Culture, Olympics, Media and Sport secretary Jeremy Hunt thinks the 64% of those questioned who believed taxpayers were paying too much would be lower if the same poll were conducted after the Games.
"It's like asking someone if they've spent too much on their Christmas dinner after the big Christmas shop, but before Christmas Day," he said.
"The time to ask whether it's been worth it all is after the event, when people can look at this amazing summer that we're going to have."
The results showed that the further people lived from London the more they thought taxpayers had paid too much towards hosting the Olympics.
In Scotland, 69% of those asked agreed taxpayers had paid too much and only 18% of people disagreed with the statement.
The bigger figure compared with 63% in the South East, 63% in the Midlands, 62% in northern England and 68% in Wales and the South West.
The poll also showed the further people lived from London the less they thought the Olympics would benefit their area.
Only 16% of people questioned in Scotland thought the Games would benefit them, compared with 37% of those in the South East, 31% in the Midlands, 23% northern England, 19% in Wales and the South West.
The opening ceremony will be held on 27 July in the Olympic Stadium in Stratford, east London, following a 70-day torch relay around the UK.