BBC BLOGS - dot.Rory
« Previous | Main | Next »

Pollsters, bookies, and boffins

Rory Cellan-Jones | 12:22 UK time, Wednesday, 5 May 2010

Who is best placed to predict the outcome of the general election: the pollsters, the bookies or the boffins - by which I mean a new breed of computer scientists who have tried to analyse sentiment in online discussions and draw conclusions from it?

Graph showing BBC poll of polls results for 5 MayThe pollsters have done pretty well in predicting recent general election results - at least since 1992 when they underestimated the Conservative vote by a spectacular amount. This time, though, they are in an extremely nervous state of mind becuase of the unique nature of this campaign. Its closeness, and in particular the emergence of the Liberal Democrats as a significant force, are making it very hard to translate headline poll figures into seats. Nevertheless, the polls have nearly all been predicting a hung Parliament for some time. The latest figures in the BBC's poll of polls are Conservatives 35%, Labour 29%, Liberal Democrats 27%, and others 10%.

Feed that into our seat calculator, which assumes a uniform national swing (but not any differential swing in marginal seats which is quite possible) and it gives this result: Labour 272, Conservative 270, Liberal Democrats 79, and others 29. In other words, something close to a dead heat between Labour and Conservatives, with neither able to form a government without a lot of horse-trading.

Turn to the bookies, and you get something rather different. Online bookmakers Betfair showed me a chart which tries to track the way the mood of punters changed over the campaign, with a hung Parliament becoming the favourite in the second half of April, then a Tory overall majority moving into the lead last weekend, before slipping just behind no overall majority again in the last couple of days. But the company, which is taking bets in every constituency, has used that data to produce this forecast of the shape of a new Parliament: Conservative 325, Labour 212, Liberal Democrats 86. That would leave the Tories just one seat short of an overall majority.

Betfair graph showing how betting has changed over course of the election campaign

Ladbrokes is also offering odds on every constituency, and it too predicts that the Conservatives will be the largest party, just a little short of an overall majority. Its prediction for seats is: Conservative 313, Labour 212, Liberal Democrats 88.

Both firms claim that by looking at the weight of money in individual constituencies they can make a better fist of predicting the election than pollsters who assume a uniform swing.

And now to a third category of soothsayer from the new enterprise of sentiment analysis - or "voodoo" as one distinguished pollster described it to me. To be fair, most of the companies which have been analysing the mood of users of sites like Twitter and Facebook have been careful not to go as far as predicting the overall result. But Tweetminster, the firm which tracks political tweets, has really stuck its neck out. It's conducting an experiment to work out whether activity on Twitter correlates to electoral success. As of 4 May, it's found 433 constituencies represented on Twitter, and its latest prediction gives the Conservatives 35% of votes, Labour 30% and the Liberal Democrats 27%.

Tweetminster assumes a uniform national swing and so, like the poll of polls, makes Labour the largest party, but hedges its bets by predicting various hung Parliament scenarios. The organisation is very keen to stress that it's not trying to compete with pollsters, merely trying to work out whether there is any connection between word-of-mouth on social-media sites and election results.

By Wednesday evening there will be more polls to digest, more data from the bookmakers - and perhaps more analysis of Twitter "buzz" from Tweetminster. But of the three groups it's the polling organisations which will be waiting most nervously for the results. They are still generally accepted as the most reliable predictors of the election race - but could the bookies come through on the rails this time?

Comments

  • Comment number 1.

    You're overlooking the category of (self-styled)statistical gurus who are arguing against using uniform swing in allocating seats. See in this respect the very interesting debate on three different models at www.fivethirtyeight.com.

  • Comment number 2.

    My prediction for Thursday:

    The politicians will get back in, just like they always do.

  • Comment number 3.

    something close to a dead heat between Labour and Conservatives,

    I don't knwo how you can say that and keep your lunch down. If we really do get the tories winning by 6% of the popular vote and still "losing" I would expect even the BBC to declare the result invalid. We'd call it invalid in Iran or Zimbabwe, so why not here?

    Anyway, I don't think that's going to happen. I reckon the Labour vote will collapse on the day. I see Dave with 350-360 seats.

    Let's see who's right.

  • Comment number 4.

    Betfair may be pointing to a larger margin for the Tories but I imagine that the demographic of people placing bets there is not representative of the nation as a whole. In particular I think more users would be traditional Tory voters and since most people will bet the same way they vote this will skew their results that way.

  • Comment number 5.

    The other category of bookies is the spread-betting companies. Their quotes for the number of seats won by each party is very close to the Betfair predictions quoted in the article. I think this view of how the betting markets see the likely result is a major omission from the discussion of the current state of the polls by the news media. This view is very different from the "neck and neck" picture portrayed by news media. Do news organisations consider the idea of betting to be slightly vulgar? Nevertheless these predictions are far more likely to be correct than the "constant swing" predictions.

  • Comment number 6.

    The pollster are in a right lather. They know that people aren't always honest about who they'll vote for, and some who say they'll vote won't. The "nasty party" tag the Tories have in some quarter means that it's the Tory vote that is under-reported by the pollsters - the pollsters know this and attempt to compensate, but it's an inexact science. The emergence of the LibDems as a force has also knocked their analyses into a cocked hat - as saying you'll vote LibDem is fashionable - but no one knows if these statements will turn into votes in the polling booth.

    My money's on the bookmakers. When people place a bet, they are putting their own money where there mouth is. They are reading the polls, talking to friends, and coming to their own judgements. Right now, SkyBet (other bookmakers are available) are offering 5/6 on a hung parliament, and the same 5/6 on a Conservative outright majority.

  • Comment number 7.

    This is the first election - US or UK - in over a decade I haven't bet on. Too close, too many uncertainties. Individual constituencies seem to have more localised battles as well, which make national swings less important.

    Also, with the plethora of digital media, in addition to newspapers and many TV channels, there's just too much information, bias and opinion being thrust in every voters face. Trying to get information on e.g. immigration, as the number 2 issue, and there's a tsunami of conflicting views, facts, figures, opinions, solutions, definition of the problem (if there is one). This seems to be the only website which offers details on this particular issue in a format suited to today's busy digital/traditional information consumer:

    http://www.isbritainfull.com/

  • Comment number 8.

    I fear the bookies are probably right - you never meet a bankrupt bookie!

  • Comment number 9.

    I downloaded and analysed the 2005 election results from here. It was very interesting for an excel-junkie like me.

    The bottom line was that my scenarios looked much like the bookies predictions and a lot less like the swingometer results. Why?

    The "uniform national swing" really ignores safe seats where a donkey with the right colour rosette would get elected. Assuming that there is going to be the same swing from Lab to Con in Bootle (Lab 75% in 2005) as in Newbury (Lab 6% in 2005) is going to lead to mistaken conclusions.

    In fact the uniform national swing would leave Labour with a negative number of votes in Newbury which the people there might think was about right but the rest of us would consider odd.

    The point is that the switchers are much more likely to act in seats where switching makes a difference - if you were in Bootle and wanted a change, would you bother voting? Same question if you were in Newbury and wanted to keep Labour in...

    Of course neither of these seats is going to switch so there is little incentive for voters there to get out and force change / defend against change.

    The bottom line is that swings will be much bigger where there is a real contest, leading to many more seats changing hands that the national swingometer suggests. This doesn't mean that the overall shares will be much different from the poll of polls which can be entirely consistent with massive swings in one place and next to nothing in places like Bootle and Newbury.

    The elephant in the room is the turnout. It was around 60% last time and I have seen nothing in any of the polls, news or analysis that suggests that people understand what it is likely to be this time. Turnout is also directly correlated to the closeness of the contest in any given seat, again because people feel that actually it is worth voting because their votes might be decisive.

    Labour actually had 22% of the possible votes last time - not 36%.

    Is there enough of an incentive for people who didn't vote last time to get out this time? Perhaps yes, given what is at stake now. If turnout goes up, where do the new votes go? If it goes down, who does it take votes away from (and will it matter anyway in those constituencies?)

  • Comment number 10.

    Opinion polls are only a 'snap-shot' of how people are thinking at the time. Many are influenced by who or what they have just seen on TV. Others like to tell pollsters the answer they think they will like.

    With the current voting system many areas vote in their own distinct way. Some are long time Labour; some Conservative; some Lib-Dem. There are also a number of marginals. So you have to get down to individual constituency level to make any kind of prediction from the polls.

    At that level even then there are a number of factors. How many parties are standing for election? Does an individual candidate have local support? Has one party been more active recently than another? What about the expenses scandal? The economy? The NHS? Education?

    There is one factor that I, personally, have not heard discussed. How many people who voted Labour in the last elections, voted for Tony Blair and not necessarily for the Labour Party? How many will change their vote from Labour to another candidate simply because Gordon Brown has been in charge and not Tony Blair?

  • Comment number 11.

    I am coming to the conclusion that the BBC is wedded to the uniform swing model because they're not capable of understanding anything else - for surely in the current environment with significant volatility, three-party politics and about 10% plumping for the "others", the uniform swing model (developed in the 1950s for a stable two-party swing) is bust.

  • Comment number 12.

    Re #4 - I would have thought (looking at the average punter) that the demographic of the average person placing bets would be skewed the Labour way rather than toward the Conservatives, leading to an opposite result...

  • Comment number 13.

    Re #1

    The debate between fivethirtyeight.com and politicshome.com suggests that models that veer away from the standard UNS model being utilised by polling companies may yield more realistic results for a 3-party race than the usual 2-party elections we have been accustomed to.

    Without more information about the bookies model, such as whether they have a weighted system based on numbers of bets in each constituency adjusted for past results, it is fanciful to suggest that the bookies have a better forecast than anyone else. One thing is for sure, the first past the post in making money will always be the bookie!

  • Comment number 14.

    Some intelligent comments here and I have to say that the spectacular simplistic way that the media analyse the polls is amazing for organisations that pride themselves on been professional


    The Beeb is the most guilty and I don't know if it's becasue they have to appear to stay neutral, but to ignore these very important facts about the Tory vote, is crass neglect for a news organisation

    1) The Tory vote is virtually always underestimated. Even in the massive Labour landslide of 1997, Conservative got more than estimated (mind you 5% instead of 3% is nothing to brag about!!). It is none as the "Shy Tory Factor" and is usually at least 2%, i.e you can add 2% onto the average of the polls and be pretty certain the Tories will get that.

    The reason for this is that a 'dyed in the wool' Labour voter gets more kudos than a similar Tory one, as it's percieved that to be a Labour supporter shows how caring you are. Hence, numerous people who say they are neutral will tend to really be Tory voters


    2) Pollsters have showed that the Conservatives have had a consistent 7% swing in all the key marginals all through this year. This shows that their vote will be higher where it NEEDS to be. Remember, the last election was an underestimated success for Michael Howard, because he narrowed the gap greatly to such a degree there are numerous seats, within such a swing, to be gained


    I think it will be a Tory majority of 15-20

  • Comment number 15.

    I can see why the Lib/Dems want electoral reform, if seats were allocated on the percentage of votes received then the results would be.

    Tories - 35% - 226 seats
    Labour - 29% - 187 seats
    Lib/Dem - 27% - 174 seats
    Others - 10% - 65 seats

    The pollsters got it wrong in 1992 because they had not taken into account the number of disenfranchised (due to non-payment of the poll tax) voters who would have voted against the Tories.

  • Comment number 16.

    Bookies set the odds so that no matter how much money is bet they make a profit. The Statistical odds on tossing a coin are evens. However if lots of money is bet on "heads" then its odds go down and the odds on "tails" goes up. The statistical odds do not change.
    It is the same with election betting. If lots of money is placed on one outcome then its odds go down. This doesn't change the likelyhood in the polling booth. So basically the bookies prediction is based on money bet. So it is exclusively based on people who bet the most. They have also been running a book since Cameron was a "cert" so still have loads money for the conservative party from several months ago. These factors gives their result an inbuilt Tory bias.
    Their odds on the number of labour seats are far too generous as they will discover when they give me lots of free money on Friday !

  • Comment number 17.

    The Figures you demonstrated today for tweetminster "Buzz" probably include the thousands of tweets today on the #imvotingconservative which was for a while the top trending hashtag. A closer inspection showed the majority of the tweets were hilarious sarcasm ie:-
    Digi_UK #imvotingconservative as Murdoch know's what is best for all of us!
    mickeast1983 #imvotinglabour and #imvotingconservative and #imvotinglibdem because i have a split personality disorder.
    dodgyscouse #imvotingconservative to increase homelessness and recycle more cardboard
    Made for a fun afternoon and demonstrated the power of SN sites.

  • Comment number 18.

    One constituency that illustrates why applying a uniform national swing is a very poor indicator of the final outcome is Torbay. In 1997 a swing way below the average national swing to the Lib Dems saw the Lib Dems scrape the narrowest of victories in a seat where based on other results would have seen a comfortable Lib Dem majority. In 2001, the Lib Dem MP increased his share of the vote massively, gaining over 50% of the vote with a swing that if repeated nationally would have seen swathes of seats, particularly across the SW change hands from Cons to Lib Dems. It didn't happen. In 2005, the Lib vote share fell dramatically with a much smaller majority leading Ivor King to claim on Election Night Special on the BBC that the result was "very bad news indeed" for the Lib Dems and would see many seats being lost to the Conservatives. Again it didn't happen - the learned professor completely failing to spot a significant rise in the Labour vote - completely out of kilter with the average national swing. Local factors will undoubtedly wreak havoc once again with the predictions of the experts - and this time around there will also be the expenses scandal impacting on some results where the sitting MP's expenses were portrayed badly.

  • Comment number 19.

    Rory, great article and some interesting stats - what is the label on the y-axis of your Betfair chart though... not quite sure how to interpret it.. %Change or %Chance? In any case, my money is always on those who deal with it so my 2 pence goes with the bookies!

    By the way, I found a 100% apt hung parliament cartoon on a South Africa news blog of all places - it has 'lighter side' interviews with everyone from the Tories and Labour to the Green's Caroline Lucas and the BNP - the punchline seems made for this post!

  • Comment number 20.

    Rory, I think Tweetminster's view will prove accurate but my money is still on a hung parliament at this rate. None of the statistics are showing anything definitive to the contrary.

    ps: @Sizwe M (#19) - LOL loved the cartoon!! Wish they had included UKIP though

  • Comment number 21.

    Re #11 - exactly right, it would take a more sophisticated mathematical model to take into account all the various permutations on people's votes given that they are likely to vote tactical more in this election than any in recent memory as there is now perceived to be a real choice for 3rd party (Monte carlo simulations, anyone? :) )

    Also, agreed with the comments on the simplistic nature of sentiment analysis and how it can be difficult to draw proper conclusions on this. More discussion on this can be seen below, which shows how poor they are at the moment

    http://www.semanticweb.com/news/are_mark_zuckerbergs_ears_burning_159528.asp

    I've been trying to analyse the data in a more contextual way using prototype tools, you can see the results on electiontrends.blogspot.com, and the preliminary results show a number of themes coming through: tactical voting and hung parliament are strong themes, with Tory and Labour leaders both being strongly attacked by followers of the other party

    In any case, it is clear to see that almost all tools are in their infancy and there's plenty of room for improvement for all of them!

  • Comment number 22.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • Comment number 23.

    This comment has been referred for further consideration. Explain.

  • Comment number 24.

    I think tweetmaster figures are going to struggle just because tweeters are a very distinct demographic.

    The interesting thing for me is going to be how soft the Lib Dem vote is. The Lib Dems have piced up a fairly fickle demographic so it's going to come down to:

    1. If they vote at all
    2. If they get cold feet and go to one of the 'safe' parties (I think Labour will pick up some votes if this happens)
    3. If the undecideds suddenly vote Lib Dem just because they are tired of the other 2.

    One thing I'm sure of is that I don't think it's going to match the pollsters predections.

  • Comment number 25.

    #3 "I don't knwo how you can say that and keep your lunch down. If we really do get the tories winning by 6% of the popular vote and still "losing" I would expect even the BBC to declare the result invalid. We'd call it invalid in Iran or Zimbabwe, so why not here?

    Anyway, I don't think that's going to happen. I reckon the Labour vote will collapse on the day. I see Dave with 350-360 seats.

    Let's see who's right."

    Well... I hope you didn't bet too much on Dave getting 360 seats.

    While I agree that its strange that Cameron could 'lose' despite getting 6% of the vote more you're looking at it from a very Tory-centric viewpoint. The way I see it is that nearly 70% of the country DIDN'T vote for Cameron. Unlike Zimbabwe or Iran we neither directly elect our Premiers nor do we only have 2 serious candidates running. We have a 2 horse system for a multi-party state. Either way a new 'Reform Act' is required because our current system isn't working too well. Worst of all we'll have to go through the same pantomime before Xmas and I doubt if the result will be significantly different because of the way the system works.

  • Comment number 26.

    Interesting analysis Rory - hope you'll be gathering some post-election sentiment analysis from the bookies too... as always they are on the money!

  • Comment number 27.

    I fear the bookies are
    probably right. you never meet a bankrupt bookie!

 

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.