« Previous | Main | Next »

Long range forecasts: The day of judgement nears.

Paul Hudson | 17:42 UK time, Tuesday, 29 March 2011

In the last two days the press have been full of stories about how we are in for a scorching August.

Of course, we have been here before. The Met Office may for ever be associated with the phrase 'barbecue summer' following its disastrous forecast for Summer 2009, which amongst other things led them to stop publishing long range or 'seasonal' forecasts.

But the Met Office is often between a rock and a hard place. Greeted with shrugs when they get it right, and outright derision when they don't, they are under far more scrutiny than any other forecasting organisation.

Take for example Positive Weather Solutions (PWS). Last summer we were told that average temperatures would be hotter than those during the summer of 1976. And although summer started well, in the end, in large part because of a poor August, summer was statistically nothing more than average.

And then yesterday, PWS is widely reported in the tabloids saying that although this summer will be mixed, August will be a 'scorcher'.

Curiously, in all the newspapers I have read, it says that 'PWS forecasts have proved to be more accurate than the Met Office' without for some reason pointing out that last year PWS got it completely wrong.

But back to the line that PWS are 'more accurate' than the Met Office. Have we got independent statistics to back this claim up? Sadly not.

It is a curious part of the 'weather industry' that all forecasting organisations that I know of, including my former employer, The Met Office, verify their own forecasts and work out how accurate they are.

Far from me to say that this could be open to abuse - but it's clearly unsatisfactory that we have no independent way of verifying the claims of all these forecasting organisations.

And so perhaps a more accurate way would have been to say that 'in Positive Weather Solutions opinion, their forecasts have proved more accurate than the Met Office.'

But help is at hand to rectify the situation. After over a year in the planning, my colleague Roger Harrabin is about to begin the 'weather test' which you can read more about by clicking here

The process aims to independently evaluate the level of accuracy of all medium range forecasters (who chose to take part that is), and will be in association with, amongst others, our very own Leeds University.

It will be fascinating to see the outcome.

Follow me on twitter @Hudsonweather


  • Comment number 1.

    About time

  • Comment number 2.

    I have no faith in the forecasts of either PWS, AccuWeather, the Met Office or any other entity that purports to derive probablistic forecasts with integrity from misunderstood complex and chaotic systems. The bottom line is that we (and by we I mean all of us, including those in the industry) are incapable of solving the equations necessary for medium range weather forecasting. The uncertainties are prohibitively great and reduce confidence to an absolute minimum, rendering the dependability of forecasts to nothing.

    Despite publicly claiming not to deliver medium range forecasts, when it transpired that the Met Office had indeed served the government with its forecast, the forecast was both useless and so cleverly phrased that it protected the Met Office from being identified as wrong.

    Tens of millions of pounds worth of Met Office computing power served up: a 3 in 10 "chance" of a warm winter, a 3 in 10 "chance" of an average winter and a 4 in 10 "chance" of a cold winter. That was it; the Met Office's contribution to winter preparation. No explanation was provided to the government on how to interpret these numbers, ensuring that if it turned out to be an average or warm winter, the Met Office could claim a ~60% accuracy (3 in 10 plus 3 in 10 equals 6 in 10) and if it turned out to be a colder than average winter, the Met Office could point out that its 4/10 prediction of a cold winter was greater than either its warm or average winter forecast.

    I find this kind of playing fast and loose with numbers, at such inordinate cost, in these times of austerity to be simply unacceptable. With this kind of performance, the Met Office shows itself to be "neither use nor ornament". It's a waste of money.

  • Comment number 3.

    Bill Foggett from the 1970s if you are as old as me and remember him on Yorkshire Television's weather bulletin - never got long term forecasts wrong - fact! Famous for getting seasonal clues from the humble 'Acorn'. Predicted the long dry spell from 1975 to 1976 with breath-taking accuracy! Died in 2004 age 91 he was a keen amateur weather forecaster who used the elements he could see and touch over many years to give his forecasts almost till his death in 2004!

  • Comment number 4.

    I guess the thing is that if the Met Office continues to make these predictions then sooner or later they are going to get one right :)

    And when they do...boy...just watch all the chicken littles come out of the wood work and proclaim the end of the world due to Mann Made Global Warming (tm)!



  • Comment number 5.

    Ironic when Met boss Vicky Pope was snowed in at Stansted airport

    Doubly ironic that she was going to a conference about global warming.

    Triply ironic that she was going to tell everyone that flying to hot places was a very bad thing.

  • Comment number 6.

    Guy Forshaw. I totally agree with you about Bill.

    The Met Office can't even get the weather right on a weekly basis, never mind annually. Any body who could believe their claims on Global warming concerns me. It is nothing more than natural fluctations in nature that we can't control.

    I thought the Conlibs were talking about reducing quangos and Paul seems to be talking about creating another.

  • Comment number 7.

    I think people are being a bit selective in their memory of old Bill Foggit. He did score some great wins and got notoriety for it, but I can remember many times his forecasts did not come to fruition either.
    Bill worked by comparing weather records and looking for patterns from the past to predict the future. In a chaotic system like weather this only has limited effect, but it may have inadvertently pulled out patterns like El Nino, La Nina, PDO, AMO, ENSO, etc. before anybody knew what they were. In this way it meant he stood a better than average chance of being right.
    My punt for this summer in my opinion ;), is for a tendency for blocking patterns to re-establish. If they sit to the west this could mean cool weather, but if they drift over us there could be periods of stunning weather. Interspersed with this will be standard mobile westerly patterns. So, basically, a British summer awaits.

  • Comment number 8.

    I find it interesting that PWS has had to issue a statement about the summer forecast, as a result of the interpretations published in various newspapers.
    Clearly Jonathan Powell is perplexed at how the newspapers have arrived at their interpretations, based on the actual forecast published.
    The answer, of course, is quite simple. It is because the way the forecast is published is so vague that it is open to such alternative interpretations.
    Best of luck to Roger Harrabin in measuring the accuracy of such forecasts, since they consist mostly of generalisations, with no numerical values. I presume that PWS are using some sort of model, in which case they must have numbers, so why not publish them? What I would like to see from a forecast would be, at the very least, numerical anomalies from normal values, for temperature, rainfall and sunshine, preferably for individual weeks. Only then could a formal evaluation of accuracy be done. Only if all forecasts were in the same format, could some meaningful comparison be made.
    Oh, and no probability figures please!

  • Comment number 9.

    The Met Office publishes a graph showing forecast accuracies, e.g.


    When I asked them for the base data this was their reply.

    The Met Office does not hold the data used to produce the graph in its original form. The original data is subject to quality control procedures which has changed the underlying data used to make the calculation. To provide assistance we could consider extracting the data which is currently recorded, however this would not give the information that you require to produce accurate calculations,

    It you do wish to obtain this different underlying data, we may need to consider whether the Regulation 12 (5) (a) EIR applies: Confidentiality of commercial or Industrial information where such confidentiality Is provided by law to protect a legitimate economic interest.

    Doesn't this sound familiar?

    Have asked again for this information, surely if they don't have the information they shouldn't publish the graph?

  • Comment number 10.

    #9 - oldgifford

    I have met with a similar response when requesting information on whether/how accuracy of 5 day forecasts was done at the M.O. Despite claiming "transparancy" over accuracy, in practice it is anything but transparent. I did eventually receive a reply however and I include some quotes below:
    "First of all I hope it is reassuring to know we agree that our current verification web pages provide an inadequate level of detail and we are sorry that these do not seem as transparent as they might be for public consumption with understanding. These pages are in the process of being reviewed and updated to provide more information to the public regarding our indices, including descriptions of how they are calculated.
    We do maintain accuracy statistics for our 5 day forecasts although we do not routinely correlate accuracy with weather type. The most useful measure of day 5 accuracy is the errors in the surface pressure pattern and we have those statistics going back 40 years. The attached plot shows that on this measure our improvement in skill could be expressed as "1 day per decade", ie day 4 forecast skill today matches that of day 1, thirty years ago. (The plot covers "area 2" which is an area-definition covering much of Europe, the North Atlantic and some of North America).

    With regard to your site specific query - our current measures aggregate 109 quality controlled UK sites. This increases the sample size to paint a more rounded view of our model accuracy. Accuracy figures are not currently produced for individual measuring stations but the data is available to produce these. There would however, be a cost to this operation.

    Forecast locations do not always correspond to observations but the "nearest observing site" should be listed on the page containing the forecast.

    Index targets and achievements are not intended to convey a high degree of accuracy but rather monitor the performance of the model and to meet the in-year agreed targets.
    The UK temperature plot at link: http://www.metoffice.gov.uk/about-us/who/how/forecast-accuracy/temp shows the percentage of maximum and minimum temperature forecasts that are "correct" using the rules below.
    "The next day maximum and minimum temperature forecasts are considered correct if they are within one degree of the observed max or min and the following day max and min temperature forecasts are considered correct if they are within two degrees of the observed max or min".
    Our scores are computed over 109 stations (as mentioned above). This enables us to get a good sample size to create the scores from. We do not store for individual sites."
    Otherwise, it appears that the M.O. don't know how accurate their 5 day forecasts are for individual sites. It is difficult for anyone else to check this, due to the fact that forecast site and measuring sites are not the same.

  • Comment number 11.


    Be intersting to see how accurate Fish was, considering he did the work himself, and how accurate the Met Office is, what with its tens or hundreds of millions of pounds in funding.


  • Comment number 12.

    10. QuaesoVeritas wrote:"accuracy figures are not currently produced for individual measuring stations but the data is available to produce these."

    But they told me they don't have the data!

  • Comment number 13.

    The Weather test sounds like a really good idea. I do wonder wether the BBC forecasts could introduce some form of 'confidence' factor for forecasts, that would let viewers understand when the forecast was likely to change or more likely to be 100% correct.

  • Comment number 14.

    I tend to agree with Paul Hudson's comments re: the Met Office can't win. For reasons that seem to have something to do with their being publicly funded, there always seem to be clouds of vultures constantly circling their long range pronouncements.

    I guess they are often open ended because the Met actually understands that long range forcasts are a dodgy business. Yet under pressure to please their masters, they have to say something. The public by contrast, seems to believe that if you put what they percieve as a lot of (their)money into it - it aught by neccessity to be dead accurate!

    As for these other forecasters, I would bet they are no better - probably worse.
    But of course, the press love a little hero who "gets one over" on officialdom. They only have to be right once or twice and all their errors are forgotten.

    I am old enough to remember Bill Foggitt. My memory is of him always predicting "a scorcher" or a bad winter due to abundant hawthorn berries or somesuch (which any botanist will tell you is due to the quality of weather the previous flowering season).

    But far be it from me to criticize people's personal religion.

  • Comment number 15.

    Sorry, I meant to post this here.

    The March UAH anomaly will be published in a few days.
    So far, the cumulative AQUA CH5 14000 ft/600mb anomaly for March, up to March 28th., is -0.24c, and I estimate that the final figure will be about -0.23c.
    Based on previous March figures (2003-2010), this is equivalent to a UAH anomaly of about -0.11c, compared to the February UAH anomaly of -0.02c.
    Converted to the HadCRUT3 base period of 1961-90, the estimated March UAH anomaly would be about +0.14c.
    While the AQUA CH5 temperature continues to be below normal, there are signs that it is catching up on the normal figure, which if continued, could produce a normal or above normal figure in April.

  • Comment number 16.

    The Met. Office five day forecast invariably changes after 2-3 days so what hope have they for 6months ahead. None at all.

  • Comment number 17.

    The N.E. got about 2 hours warning of strong winds today, even though in theory the warnings go 5 days ahead.

  • Comment number 18.

    16/17 # above

    Just goes to show how dynamic the weather can be, I guess.

    Incidentally, I do remember an accurate long range lucky guess/ forcast back in 1969.
    The Daily Express ran a contest between a weatherman (called David Bowen, I think) and an astrologer (name lost). For the prediction between June and October, Bowen was remarkably close- even to the wonderful October which proved to be the finest for centuries I believe. I don't know how he did it.

    The astrologer was completely wrong, thus proving I suppose, that whatever influence our own star, the Sun, has or has not on our immediate weather, the other stars have no influence at all!

  • Comment number 19.

    Re #14. At 22:05pm on 30th Mar 2011, jkiller56

    "I tend to agree with Paul Hudson's comments re: the Met Office can't win. For reasons that seem to have something to do with their being publicly funded, there always seem to be clouds of vultures constantly circling their long range pronouncements".

    I feel that a lot of the criticism for the Met Office is that they are too closely wedded to the CAGW theory which has led them to produce too mild winter forecasts three years in a row which in turn has meant that councils are not properly prepared with adequate grit for the roads and this in turn has cost lives.

    In the run up to Copenhagen the Met produced their free booklet on Climate Change "Warming, Climate Change the facts" stating that

    "It’s now clear that the emission of man-made greenhouse gases is causing climate change. The rate of change began as significant, has become alarming and is simply unsustainable in the long-term."


    When Climategate broke Julia Slingo of the Met Office was also responsible for the disgraceful petition that asked many scientists (fearing for their jobs at the time) to sign a statement that said

    "Statement from the UK science community
    We, members of the UK science community, have the utmost confidence in the observational evidence for global warming and the scientific basis for concluding that it is due primarily to human activities. The evidence and the science are deep and extensive. They come from decades of painstaking and meticulous research, by many thousands of scientists across the world who adhere to the highest levels of professional integrity. That research has been subject to peer review and publication, providing traceability of the evidence and support for the scientific method. The science of climate change draws on fundamental research from an increasing number of disciplines, many of which are represented here. As professional scientists, from students to senior professors, we uphold the findings of the IPCC Fourth Assessment Report, which concludes that "Warming of the climate system is unequivocal" and that "Most of the observed increase in global average temperatures since the mid-20th century is very likely due to the observed increase in anthropogenic greenhouse gas concentrations"."


    Many of the signators did not even work in the field of climate change and simply would not have had the knowledge required to truthfully sign this indeed it is scarce credible that anyone could have signed a petition stating.

    "They come from decades of painstaking and meticulous research, by many thousands of scientists across the world who adhere to the highest levels of professional integrity. That research has been subject to peer review and publication, providing traceability of the evidence and support for the scientific method."!!!

    As many commenters pointed out at the time -


  • Comment number 20.

    i fail to see how we can trust any of these long range forecasts , lets face it the forecast is never that good when predicting about 5 days in advance.
    might as well stick to berries on trees and seaweed as an indicator for all the good the computer predictions give us .

  • Comment number 21.

    The Met office has one of the largest supercomputers on the planet and they still get it wrong. They want more money to enlarge this machine but will still get it wrong because their models are wrong. Garbage in-garbage out.

  • Comment number 22.

    Years ago the Science Fiction magazine ‘Analog’ ran monthly fact articles. So good that on one occasion the FBI seized every magazine for sale, including those of street sellers. The fact article hypothesised on how to harness nuclear energy to make a bomb. They thought the Manhattan project had been compromised.

    Analog also ran an article on Astrology. They cited a person making a very good living using the positions of the solar bodies to predict radio propagation characteristics, which in those days seriously affected communications companies revenue streams. They used similar techniques to compare the US weather service forecasts with forecasts based on a roulette wheel? planetary positions and from memory [ getting worse each day ] the US weather service and the roulette wheel were in pretty close agreement at around 50% whereas the astrology techniques yielded around 80+%.

    As an aside they also suggested characteristics applied to people born in certain zodiac signs may have some credence if you wind the clock back from birth to conception, suggesting that cosmic ray or other solar influences just might have some effect on the genetic make up at the time of conception.

    Unfortunately my copy of that magazine got lost in a house move. With tongue in cheek they looked at Kennedy’s birth zodiac [ he was still in his first term of office at the time ] and noted it was the same as those US presidents who had died violently in their terms of office.

  • Comment number 23.

    To Sparklet #19

    I couldn't comment on how "disgraceful" the petition you mention was. However one could read the intentions quite differently.

    Whatever the motives and wisdom behind "Climategate" (please, don't drag them up again) - as I remember- there was a general feeling amongst some of the scientific community that too much would be made this event and public faith in the science behind AGW (already weak) would be further undermined; in their view, unjustifiably.

    So, yes, you could interpret the petition as an attempt to keep the "gravy train" afloat.

    Equally it could be seen as a genuine attempt (however naive in this predatory and cynical media world we inhabit) to stem a haemorraging of faith in the evidence for AGW, which, conspiracy theory aside, does seem to suggest that the majority of scientific opinion supports one way or another.

    Since very few of us on this blog (myself included), it seems to me, really appear to have the scientific background to judge all the evidence re:AGW thoroughly and objectively (even assuming such a position is possible), it really depends on who you prefer to believe.

    I suppose as long as one is aware that this, at bottom, may be an emotional rather than a wholly rational response and can be prepared to accept that any point of view could be totally wrong, then one can wear whatever hat fits.

  • Comment number 24.

    Re #23. At 16:46pm on 1st Apr 2011, jkiller56

    No matter what the 'intentions' it seems wholly unethical to me for a senior member of the Met Office to ask scientists (many fearful for their jobs) to sign a petition they could not possibly truthfully sign. Hence the term 'disgraceful' as it is placing the scientists in a very difficult position.

    For those wanting to know more about the background of AGW then the recent testimony of Professor John Christy (Professor of Atmospheric Science and Director of the Earth System Science Center at the University of Alabama in Huntsvilleand Alabama’s State Climatologist) makes interesting reading.


  • Comment number 25.

    #18 - jkiller56 wrote:
    "Just goes to show how dynamic the weather can be, I guess."
    Precisely my point.
    If they can't forecast the wind 5 days ahead, or even 1 day, how can they forecast season's ahead, or even the climate in 100 years?

  • Comment number 26.

    The day of judgement for long-range forecasts is here. Now anyone can open a brokerage account and start trading weather derivatives (heating and cooling degree days) on Chicago Mercantile Exchange. It's not just for big corporations. An individual account with just a few thousand dollars would make it possible to buy or sell HDD or CDD futures contracts. If Piers Corbyn thinks his forecasts are the best, go ahead, open an account and start trading. If he becomes a millionaire, that would be the ultimate test for his forecasts. Put your money where your mouth is.

  • Comment number 27.

    QuaesoVeritas# 25
    Yes, but the question is - is anyone else any better?

    You must admit, that forecasting the weather in this part of the world is extremely difficult. And surely you are not suggesting that some level of inaccuracy therefore invalidates all MO forecasts? (if in doubt, try asking the shipping fraternity).

    Nor, I trust, are you trying to obliquely insinuate the populist view - that because the weather forecast is not exactly right every time - we therefore do not need to take any notice of what is said about the less comfortable aspects of climate change.

    Though I am sure future climate will spring many surprises and some of todays predictions will prove wide of the mark, it would be disingenuous to allow criticism of the general weather forecast to become a comfort blanket.

  • Comment number 28.

    Unfortunately it seems that this 'Weathertest' will contain many of the same types of conflicts of interest that bedevilled the Climategate 'Independent' Inquiries.

    "It was interesting that Harrabin referred to the Met Office’s nervousness about the project. Given that just about every outside agency and University involved in assessing forecast accuracy are Met Office partners in various meteorology and climate change, that seems to be over egging things".


    Quoting Roger Harrabin -

    "The independent Philip Eden had volunteered, and I liked this because Philip is regarded as genuinely independent.

    But then others objected that the Met Office is the official provider of World Meteorological Organisation statistics and it would be ludicrous to reject their highly sophisticated statistics smoothed by computer models just because the Met Office forecasters were amongst the contestants in the Weather Test."

    As AM points out -

    "First, with respect to Philip Eden. He is the former the Chief network weather presenter on BBC Radio 5 Live who is now Director of the Chilterns Observatory Trust. Some might question how independent he could have been considered as a result.

    Second, it is incredible that the Met Office gets to act as competitor and judge in the same project when their forecasts are being compared

    Third, in a piece titled ‘Understanding Climate Change’ BBC Devon & Cornwall announced last week that:

    The Met Office and Exeter University are to form a world class academic partnership to tackle the problems of climate change.

    Along with two other British Universities they’ll try to understand the impact of extreme weather.

    The other two universities, we learn from watching the video clip, are… Reading (former home of the Met Office’s Julia Slingo) and Leeds – the statistical judges of the Weather Test (mentioned in Harrabin’s email further above). We only need someone from Exeter added to the management of the project to complete the climate change advocate set. Surely Harrabin knows Leeds and the Met Office are partners, so how can Leeds continue to be involved?"

    So to sum up, we have a mutually supportive corporate relationship between the BBC (whose project the Weather Test is) and the Met Office, who act as both competitor and judge. We have the Met Office’s mutually supportive new world class academic partnership, which includes Leeds university who are part of the project management. We have the Met Office’s unofficial PR man from the BBC, Harrabin, leading the project. And that project leader has a sideline in speaking at or chairing meetings of climate change advocacy groups who share the Met Office stance on AGW"


    So it seems the Met.Office along with its partners are going to judge its own performance using 'sophisticated statistics', 'smoothing' and 'computer models'.



  • Comment number 29.

    Re #26. At 01:13am on 3rd Apr 2011, user010 wrote:

    "The day of judgement for long-range forecasts is here. Now anyone can open a brokerage account and start trading weather derivatives (heating and cooling degree days) on Chicago Mercantile Exchange. It's not just for big corporations. An individual account with just a few thousand dollars would make it possible to buy or sell HDD or CDD futures contracts. If Piers Corbyn thinks his forecasts are the best, go ahead, open an account and start trading. If he becomes a millionaire, that would be the ultimate test for his forecasts. Put your money where your mouth is."

    Perhaps he lacks that 'get rich quick on the backs of others' mentality so prevalent in those leading the CAGW agenda.

    Gore certainly enjoys the fruits of his labours but odd that it's so near the ocean he was so concerned about in 'Inconvenient Truth'


    And he tends not to bother so much about his electricity consumption...


    Ah, but then he purchases carbon offsets (from himself of course) to make up for it!!! ...


  • Comment number 30.

    Sounded like a good idea till I read that a certain Mr Harrabin will be overseeing the exercise

  • Comment number 31.

  • Comment number 32.

    #31 - brossen99 wrote:

    I am afraid I am not very impressed by Piers Corbyn's forecasts.
    As far as I know, he predicted major earthquakes and extreme weather at the end of March, due to solar activity, but didn't specify where. If there weren't a few earthquakes and extreme weather SOMEWHERE in the world, it would be most unusual.

  • Comment number 33.

    I'm also in agreement with Paul about the public perception of the Met Office. When they get the forecast correct, nobody bats an eyelid, but when a forecast goes wrong they are pilloried in the press, and Jonathan Powell and his PWS organisation jump on the bandwagon. I see PWS have a response to Paul's article on their website which to be fair has a reasonably balanced view.

    However the main points of their summer forecast which were a potential for localised storms / flooding in June and prospects of a dry August were stating the obvious. I suspect storms and flooding occur somewhere in the country every June, and perhaps the forecast of a dry August follows on from last years forecast on the basis if they keep forecasting a dry August, it will eventually happen.

    Secondly on a point of fact, the Met Office does not have a monopoly in the UK weather services market. Yes, they employ staff who appear on many of our television screens, but almost all of the forecasts you see in the press are produced either by Meteogroup or Accuweather, and their forecasts are not worth the paper they are written on. Unfortunately the Met Office still get the blame for these poor forecasts because the Meteogroup logo is hidden away at the bottom of the page in small print.

    The ambition of PWS to compete for business against the Met Office is also an interesting concept. I wonder how impressed the RAF would be if they received some of the vague forecasts PWS publish ? The RAF deal with cloud bases, visibility and winds to the nearest knot, and not just dry with sunny spells, which interestingly was the PWS forecast for Northern England on Monday and it rained !


BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.