Rory Cellan-Jones

Google's 55 minute crisis

  • Rory Cellan-Jones
  • 2 Feb 09, 10:21 GMT

It was the weekend which shook Google to its very foundations... possibly. All around the world, users of the search engine which prides itself - and has built a multi-billion dollar business - on providing the best, most useful answers to every query were greeted with a message warning that "this site may harm your computer".

The same text was pasted next to every search result - for everything from Aardvarks to Zoroastrianism. In effect, Google was warning users that the entire internet was sick and shouldn't be touched with an electronic bargepole.

Panic spread as the global web community told each other about this apparent breakdown. Google eventually fixed the problem, explaining that an update of a list of malicious sites somehow ended up including every web address, instead of one specific URL. But what a textbook example of a corporate disaster allowed to rage out of control...

...hold on a minute. The entire incident lasted just 55 minutes. And if you were on the West Coast of the USA, where Google is based, you are unlikely to have been affected, because it started at 0630 your time on a Saturday morning and was over by 0725. So why all the hoo-hah?

Well, news spread like wildfire around the blogosphere - or more accurately the Twitterverse - as everyone seemed determined to pass on their panic, along with rumours that Google's Gmail was also misfiring. So far. more people knew about the "may harm" incident than were affected by it. I got off a plane on my way home from a holiday on Saturday afternoon, asked online whether I'd missed any big tech stories, and was bombarded with messages about the Google crash.

screengrab by bill thompson

So what's the verdict on the importance of this incident? One blogger decided that Google could have done better, awarding the company just 5 out of 10 for its handling of the crisis and accusing it of failing to respond as quickly as it could have done. That seems a bit harsh - it sounds like someone was hauled out of bed in Mountain View at 0645 on a Saturday to tinker with the engine, and got it sorted straight away. But it does go to show how rapidly you need to react to a crisis in a world where your mistakes will be the subject of web chatter within minutes of them happening.

Two Twitterers gave rather different views of Google's 55 minute crisis. One said, perhaps not completely seriously: "kennedy, lady di crash, and now #googmayharm! Everyone will remember what they were doing". Another put it like this: "small message under Google search results brings Twitterverse to a halt. Real world curiously unaffected!"

And the other thing to remember is that while the web's reactions may be fast, and its gaze intense, its attention span is very short. In Britain at least, a temporary problem with a search engine has been eclipsed by the arrival of snow. #uksnow is the tag attracting thousands of comments on Twitter - and I suspect it may take a little longer than 55 minutes to sort that out.


  • Comment number 1.


    Shallow is a very appropriate word here.

    Shallow view of the world - and to be honest - quite shallow snow considering the "mayhem" and "hysteria" being caused!

    I twitter - therefore I have a smartphone.

  • Comment number 2.

    Google was warning users that the entire internet was sick and shouldn't be touched with an electronic bargepole.

    It's certainly a point of view :-)

  • Comment number 3.

    It's quite amusing really.

    We're all human, and we all make mistakes.

    Everyone should just have a little giggle, and then move on...

  • Comment number 4.

    Google was out for 50 minutes....big deal!

  • Comment number 5.


    Ha ha.

    What this does illustrate quite pointedly is the lack of critical rigour in the dea-eyed, bovine stampedes that are the twittererse and blogosphere. It's comical to see what amounts to a hi-tech game of Chinese Whispers being reported as newsworthy.

    What actualy happened:

    For 55 minutes a (ubiquitous, admittedly) software product had a (non-critical) problem. It was fixed. end of story. and thenit was seized upon by those with little better to do that propagate it with no critical faculty.

    Can we have some sense of perspective here? Please?

  • Comment number 6.

    Part of the "hoo-hah" would have been because while it might have been early in the morning on the US East Coast, it would have been mid-morning on the West Coast & early-afternoon here in the UK.

    Another part is that Goggle does not make it easy to 'continue' onto the link you want, so many people were 'lost'. Personally, and unfortunately, even before this incident, they've had a significant number of "false positives" that have meant to me, at least, that what initially appeared to have been a useful feature was turning out to be a irritant enough to raise thoughts of alternatives...

  • Comment number 7.

    All things considered, I think Google handled this really well.

    They responded quickly, fixing the problem and posted an open and full explanation of what had gone wrong.

    There wasn't any attempt to sweep the news under the carpet, but just an apology.

    In fact, it could be argued that some good will have come from this. Imagine how many people who normally are lackadaisical about their anti-virus protection will actually have been stirred into scanning their computers by this incident?

    If it caused people to check their PCs and update their anti-virus defences, then Google's human error could actually have done all of us a little bit of good.

  • Comment number 8.

    Not much of a crisis. Also not so much a technical issue.

    I copied the "" link that was on the page all google results were directing you to and pasted into the address bar, hidden in the text was the following...


    ...It disappeared after pressing return and we could not get it to appear again.

    Technical issues or hack?

    Either way it was all very amusing. I considered the response time quite impressive really. Especially since the issue became intermittant then went away rather than just vanishing which you would expect with what Google described the issue as being.

  • Comment number 9.

    ....panic spread
    ....panic spread

    Get a grip on yourself Rory

  • Comment number 10.

    I was there when it happened. My wife complained that Google wasn't working, and I sighed the deep sigh of the digitally able.

    But sure enough, when i took control, google told me everything was toast. I use a mac, so my world was upside down. Either the mac had failed, or google had failed. Both possibilities are heresy.

    People ask what is the big deal, but I don't think they realise how much our world has changed since the internet arrived.

    "I don't know" now means "go away I am not interested".

    "Can you tell me.." now means "I've lost my iPhone", or "I'm as crazy as a hat full of snakes so look out."

    Old people must find it curious in the extreme. Myself, I can almost remember what it was like to value books, and to feel important because I had read a lot of them.

  • Comment number 11.

    Crikey - at one point I thought my Mac had a bug...
    Silly me

  • Comment number 12.

    I too was called upon by my partner to investigate what she deemed to be the likely possibility some eastern European hacking cartel had stolen her internets. Thirty seconds after booting up my machine, it became obvious Google had suffered nothing more than a human, if monumental, screw up.

    Nothing to see here you would think, and to a certain extent (other than the opportunity for amusing screen grabs) you'd be right. But I think you may be missing the point Rory, as you didn't experience it first hand. Yes it lasted no longer than an hour, but in that hour both me and the missus set our default tool bar search engine to another provider and, as typical examples of humanity, unless something similar happens again, we'll unlikely be bothered to change it back again.

    To Google, that matters.

  • Comment number 13.

    A cautionary tale showing how much we have come to rely on too few baskets for our online eggs.

    When Google went bad, I looked to BBC news and there was nothing.

  • Comment number 14.

    All it really says is that one shouldn't trust Google unconditionally. Simply because Google doesn't say a site is dangerous is no reason to assume it isn't. I will say that despite the stupidity of the problem, the staff do seem to have handled it well, ending the problem in under an hour.

  • Comment number 15.

    For another google giggle, I just installed the latest version of Earth.

    Got this message:

    The Google Earth Plugin does not yet support this browser.

    Supported browsers on Windows currently include Firefox, IE 6, and IE 7. Learn more ยป

    What browser am I using, Ooops, Google Chrome.

  • Comment number 16.

    As a professional programmer, I can attest as to how easy it is to make this kind of mistake.

    Suppose the record to be updated had an 'id' of 897345987. The SQL command to update it would be something like:

    UPDATE The_Big_Index
    SET Give_User_Warning = True
    WHERE ID = 897345987

    It is horrifyingly easy to omit the WHERE ... part, especially if you are in a hurry (and these days, as a highly paid ;-) professional, one generally is), and if you do you end up updating every single record in the table, even if there are billions of them.

    Last year I deleted an entire table in a 'live' database due to a similar aberrance. Fortunately, there was a backup taken a few minutes before, but if there had not been I would have been in it up to the armpits, as the expression goes.

    SQL needs the addition of two facilities: one is an 'idiot' flag that can be set on a database user; the other is a ROLL BACK FROM IDIOT command.

  • Comment number 17.

    "SQL needs the addition of two facilities: one is an 'idiot' flag that can be set on a database user; the other is a ROLL BACK FROM IDIOT command."

    But SQL already has this. The first is table and database level security controlled via privileges in SQL and by other means in individual products. The second is COMMIT and ROLLBACK.

    Why were you working on the live db anyway? :)

  • Comment number 18.

    I know, I used GOOGLE as my search engine on Saturday....Those messages were showing up on my computer.....

    And, even I had an email with a RED warning...I know that source of information.

    ~Dennis Junior~

  • Comment number 19.

    I got this. After a minutes I understood that it was clearly an updating error, that would be fixed. Since I have sites I Googled them to see if they were flagged to confirm, as I could independently check the files.

    Move on people, mass panic not required.


    Google works on live data because it is so distributed, updates more or less must happen live. No system is perfect, there is mathematics to prove this must be the case. We need, collectively to get over it.

    If people were that worried about perfection, they would not use Wikipedia.


The BBC is not responsible for the content of external internet sites