bbc.co.uk Navigation

Rory Cellan-Jones

Does Wolfram work?

  • Rory Cellan-Jones
  • 18 May 09, 08:23 GMT

Over the weekend one of the most interesting experiments yet seen in giving web users access to massive computing power has gone live. The official launch of Wolfram Alpha is still scheduled for later on Monday, but thousands of users have already had a chance to try this new way of extracting information from the web.

When I spoke via a video call to Stephen Wolfram, the British physicist behind Wolfram Alpha, on Friday night he was still not entirely confident that his "computational knowledge engine" would fire up. His team at Champaign, Illinois had just used one bunch of super-computers to send vast amounts of data to another, and that test had gone, in his words, "horribly".

But, despite a few hiccups when searchers got "sorry" signs, the system seems to have held up. So what's the verdict? Will it be a Google, changing the way we see the web and the world - or will it be be a Cuil, a much-hyped search engine that sinks without trace after the initial interest?

Wolfra mscale of Eb majorAfter playing with Wolfram Alpha for a few days, I've found some wonderful things, and vast areas where it is of little or no use. Type in a musical scale, such as Eb major - and it will play back the notes. Look at the weather in London - and it will allow you to compare today with the same day 10 years ago. If you're stuck on a crossword clue and enter a_a_r_m, it will tell you the answer is "anagram". But it is very dominated by US data - so it will tell me the average earnings of waiters in the US, but give me little or nothing about wages in the UK. And even in the United States it has its limitations - ask it for a list of the world's highest buildings and it quickly obliges, but it won't produce a list of the tallest skyscrapers in the US.

As well as trying it out myself, I asked three people who might well have a use for it - a doctor, an economist and an A-Level student - to have a look and give me their first impressions. I warned them that it was not the right place to go to find where the new Star Trek movie was playing (though it does list the cast) or for stories about Jordan and Peter Andre, or any number of other web searches where you're looking for sites, not answers.

The point of it is to answer any question involving data - from historical facts, to scientific knowledge, to share prices - and to present the figures in all kinds of useful ways. But I'm afraid my testers were not terribly impressed:

The doctor

This tester is a consultant at a major hospital, and he was none too impressed with Wolfram's command of medical data.

"In a nutshell, so far so bad," he told me. "I've asked it various fairly general health questions, including:
- What is the rate of MRSA in UK hospitals?
- What is the survival after stroke in the UK?
- What is the nearest hospital to Oxford?
And I've rephrased these questions several times with no answers. If I get onto my own speciality it is completely clueless. If you don't ask a question but just make a general enquiry about, say, stroke then you can get some headline epidemiological data."

However his 8-year-old daughter fared better. She asked ""How old was Tutenkhamun when he died?" and got the right answer, and, after a bit of tinkering, found out the height of the pyramid of Giza. But, as the doctor pointed out, you could get the same results from Wikipedia.

The economist

This economist is always on the lookout for all kinds of data - ranging from governemnt statistics, to company balance sheets, to figures on technology in developing countries. Here's what she said:

"The formats in which it can answer questions are rigid. The source material on which it draws seem limited - I'm very unclear what it's actually searching. Almost everything I tried got the message that it didn't understand my question. Even questions phrased just like some of the examples, with a small change, eg banker instead of forester, got that same answer. There seemed to be almost nothing it could say on my own interests - I couldn't find a way to get it to tell me mobile phone ownership in different countries, which is easily found on the ITU, UNDP and World Bank websites. Oh dear."

The school student

This 18 year old is doing A-Levels in mathematics, politics and history. He was more impressed than my other testers - perhaps because he is younger and more open-minded:

"GDP UK Germany" gets good results,"GDP UK Germany 2000-2008" does not. It also seems good at comparing certain types of data for groups of countries, eg "NATO military expenditure" gets you a comparison of how much is spent in each NATO country, and "EU life expectancy" compares life expectancy in all the EU countries.
Mathematical things seem to work the best, like "volume of a sphere radius 4.5" or "integrate xsinx" or "solve x^2 + 5x + 6".

So not a great set of results overall. But I'm not going to dismiss Wolfram Alpha's chances of success - for three reasons. First of all this, as Stephen Wolfram stressed to me, is a work in progress - at the moment it has got a lot of US data in its super-computers, but not much from anywhere else. Its cry will be "give us your data and we will set it free", so it should be far more useful in a few months' time.

Secondly, where it does work Wolfram is tremendous fun, like a big brother who is just brilliant at sums and wants to show off what he can do. I think a lot of people will visit in the early days just to play.

And finally, it would be nice to have one decent alternative to Google - even one that makes no claims to be better at search. Wolfram Alpha's arrival has already spurred the giant of search to launch a similar product, "Google Squared", which will soon emerge from its labs.

Competition - as Google itself stresses - is a very healthy corrective to complacency, and it's time we had a new kid on the block to show us a different way to use the web.

Comments

  • Comment number 1.

    After a few initial test runs of Wolfram|Alpha, I've begun to see how it can be used more in theoretical scenarios, whereas I'd use Google in real-life scenarios. Being an A-Level student myself, in Politics, German, Religious Studies and Politics, I have a use for both Google and Wolfram|Alpha.

    Initially, I found the idea by W|A hard to work out - some of the input I was giving it simply produced a message saying that it didn't know what to do with my data, even when I had inputted something fairly simple, such as, for example, "2 to the power of five".

    Another quibble I'd have with the W|A service was that - on launch and all through the weeekend - it was horrendously slow. So much so, that after a minute or two of waiting, I'd rephrase my input and take my business over to Google, where I could be guaranteed a response within less than a second.

    Thankfully, the W|A team seem to have realised that traffic was causing a big problem, and either a new server has been introduced, or they've installed some traffic load balancing gear. Either way, it's a big improvement this morning over Saturday night.

    In all, whilst I see W|A is extremely powerful, I can't help but think: does it really have a purpose outside the university physics lab?

  • Comment number 2.

    I asked Wolframalpha the question
    'How many poeple live in Wales?'

    Answer? 5956

    Not quite right...

  • Comment number 3.

    Not so much a Google, more an encyclopædia (and not a Wikipedia either)?

  • Comment number 4.

    @blognickwc Yeah, it's better with US data, to be honest. Shame really, as the web's a global community. Maybe they should introduce a spider to crawl the web for such data? Would certainly increase it's usefulness beyond the laboratory.

  • Comment number 5.

    I asked how long the longest film ever made was and got nothing after trying several different approaches to this question.

    I then tried longest book ever written, with the same results.

    In my opinion if you have to first work out how to phrase your question to get the right result, you could have found the answer with Google and/or Wikipedia.

    This site is going to have limited appeal to Scientists, Academics, Journalists (and other resaerchers seeking facts), but will not appeal on the broader market.

    I initially had concerns over the sources used to retrieve my answer, but on reviewing the sources used for retrieving data on my home town I was satisfied that this data could be relied on. And I was actually pleased to see that this data is referenced according to the Harvard referencing methodology (I believe) so this could be copied directly into any acadmenic journals and papers you are putting together.

    So, maybe not for me (unless I go back to university to do a PHD), but I can see the appeal for certain individuals.

  • Comment number 6.

    I entered "Sedgefield Racecourse" and it told me the distance from Sedgefield (Co. Durham) to Race Course (Clarendon, Jamaica) - a place I didn't know existed!
    I think you've been caught up in the hype on this. Please hold back and only bring it to our attention when it warrants it.

  • Comment number 7.

    This isn't a rival for Google because it isn't a search engine. It's a 'computational knowledge engine'. It doesn't search the web for answers, it uses it's own certified data.

    However it may eventually take a proportion of Google's traffic as people use it when a maths or statistics based factual answer is required. It's success depends on how quickly they can ramp up the amount of data in there.

  • Comment number 8.

    I'm slightly concerned (probably more so by the standard of A-level maths students than anything else) if it's answering problems such as "integrate xsinx" and "solve x^2 + 5x + 6". Unless of course it answers by pointing out the errors in the questions.

    /mathspedant

  • Comment number 9.

    I like the idea of this, it's could be something very very special indeed.

    However it's hard to break out of the "google" way of doing things, it's not very good at interpreting queries formed in something resembling natural language.

  • Comment number 10.

    well I just tried typing in hayfever and it said it did not understand and was I loking for 'Gay Feather'???

  • Comment number 11.

    I tried this and confirmed it is yet more over-hyped rubbish...

    Why doesn't the 'tech' world have a degree of healthy scepticism rather than a foolish [and somewhat childish] "Colours ! Bells ! Whistles ! " approach to anything with sound, light and motion..

  • Comment number 12.

    WA was unable to understand the name of my nearest town "Kings Lynn" no matter what syntax I used.

    If it falls down on trivial test queries, I'm not likely to bother going back with anything tricky.

  • Comment number 13.

    I asked some basic questions, and it understood the majority. 'When was Charles Dickens born?' yielded an answer, 'where is the Taj Mahal?' gave a map, a word-based answer, but also lots of interesting local data, such as population, time, local weather, and distance to Delhi. 'How far is Sirius from Earth?' succeeded fine, with the answer presented in a variety of units or measurement, and 'how big is Japan?' gave a similar result, including interesting comparisons with other countries. You could get this from other sources, but for simple facts, it might be quicker to ask WA than trawl through a text-based encyclopedia article.

  • Comment number 14.

    I read the subtitle of this blog on the BBC News homepage "Will Wolfram Alpha work as a rival search to Google?" and buried my face in my hands.

    Within a few minutes of using Wolfram Alpha the advantages of the 'computational knowledge engine' became very clear.

    To compare WA to Google is extremely misguided and simplified. If anything, comparing it to Wikipedia or the new Google Squared would be much more worthwhile.

  • Comment number 15.

    I've had a bit of a play with Wolfram Alpha and it seems to deal well with basic requests for information, however, can't really handle anything you couldn't find out from skimming the Wikipedia article on the subject.

    As a medical student I feel it would be of little use to me, I tried a few relatively basic fact-based queries and it couldn't return any results. I'm also concerned about reliability, since my first result showed up information I knew was incorrect, yet it is difficult to know the accuracy of the data, since the source is not automatically displayed. Google might take a moment longer since you have to open up specific web-pages, but at least you can tell whether the information is likely to be out of date, or written by someone who doesn't even know what scientific method is, let alone utilise it.

  • Comment number 16.

    blognickwc - sounds like they need serious work on disambiguation. One of the civil parishes in South Yorkshire is Wales (comprised chiefly of the villages of Wales and Kiveton Park) with a population of around 6000....

  • Comment number 17.


    I tried 3 very simple questions....and got the earthshattering...

    "Wolfram|Alpha isn't sure what to do with your input."

    Amazing. It's clearly so clever, it's decided my questions are too stupid to answer. I think google are safe.

  • Comment number 18.

    The search engine is sort of useful, based on my initial 'exploration'. I still wasn't able to find out how to buy foreign music (I'm trying to find a site where I can buy a year 2000 Korean song), but it was helpful and fun for looking up country statistics. However, some of the statistics are off, I suspect because some of the source information is old or faulty.

  • Comment number 19.

    I think we need to learn how this search thinks and once we do it will get easier. To those to say 'That's not the point' remember that to get good info out of Google you have to know a few tricks when searching. This will be the same and we are just going to have to learn what they are.

    The results of my queries were dire but I can see the potential so I'll come back and check again in a few months when they have worked a few more of the kinks out and added more non US data.

  • Comment number 20.

    A search for how many countries are in the world was also incorrect, with Macau, Greenland, and Pitcairn all counting as countries (and not in the the-UK-is-four-countries sense).

  • Comment number 21.

    I think Douglas Adams would approve! Afterall the Hitch Hikers Guide To the Universe taught us that it is not the answer that is important, but the question.

  • Comment number 22.

    Computers are generally bad at ambiguity. This is why most Web searches succeed after we identify some specific piece of information, to define our search. The more specific an item we are searching for, the greater the likelihood of our success. Most people search the Web by trying to second-guess the kinds of things that someone with the answers that they need might have written. When looking for a colleague's professional web page, for instance, it is often better to type in their telephone number, than their name, because the number is more likely to be unique. Human beings are often quite good at this sort of guesswork, whereas computers are monumentally bad at it.
    Google is good at second-guessing certain types of action, too (by presenting us with a map, say, when we type in something like "W12 7RJ" - on the assumption that we may be searching for a postal code), but it is only good at that sort of thing, because engineers working at Google have recognised that A) people using their search engine very often tend to put in postal codes, that B) UK postcodes tend to match a general, but highly regular, pattern, and C) someone typing in a postcode probably already has the rest of the address - so chances are, they're looking for directions. The search itself, is highly specific, but it took another group of human beings to see that it represented a pattern of behaviour, and that the behaviour itself had a pattern, that could be identified and handled.

    Wolfram falls over because it tries to guess what kinds of questions it might be asked, and encourages us to be vague; whereas Google is built around the kinds of questions it IS asked, and encourages us to be specific.

  • Comment number 23.

    This is a very useful addition to the quiver of tools web uses can employ to get information. I just had my quadratic equation solved and plotted and the chemical formula of ethane showed to me. Both of these were one click solutions - far faster than Google or dusty textbooks.
    Wolfram Alpha is designed to produce objective data so I can see why subjective questions would fail.
    Given WA's stated brief, I would say it works very well indeed.

  • Comment number 24.

    It's an unmitigated failure.

    I've tried many searches and all of them come back saying it has no idea what I'm asking, despite them being quite valid, straightforward questions to which Google finds the answers with ease.

    Search for "How many species of giraffe are there?" and Google comes up trumps first result, Wolfram just spams you a load of data about Giraffes from which you can just about extract the answer from a horribly rendered diagram.

    Well, Wolfram's known for his maths, so I figured I'd try something a bit more mathematical, how about "How many non-isomorphic trees are there with 6 vertices?", well, Wolfram just falls over flat, unable to help at all. Google struggles a little, but the first page of results can get you the answer you need.

    "How many species of Melocactus are there?" - Google answers with it's first result, Wolfram outright fails.

    "On what date was Adolf Hitler born?" - Google answers with it's first result, Wolfram outright fails.

    "What year are the next US presidential elections in?" - Google answers on the first page (but not the first result), Wolfram outright fails.

    "What is the melting point of steel?" - Finally! Wolfram answered! Unfortunately it was an unhelpful answer, whilst Google explained that Steel has no fixed melting point and it depends on the composition of the steel with it's first result.

    I'm having more trouble finding something that Wolfram can answer - maybe it's good at really simple stuff like "What is 1 + 1?", it seems to be able to do things like this, it seems to be able to do measurement conversions too, but what's the point when Google does all this and everything else as well?

    Wolfram brings nothing new, but not only that, it doesn't even bring things other search engines have had for years. It is over-hyped and effectively useless.

  • Comment number 25.

    I asked it:
    To compute a regular expression
    For the mass of alpha centauri
    For the friction coefficient of ice and aluminium

    All FAIL

  • Comment number 26.

    Yes I think that this was a valiant attempt, but it's just not cutting it.

    Who said we needed to write out full sentences to communicate to a search engine!?

  • Comment number 27.

    almost entirely useless.

    It looks like a leaked pre-release proof of concept. not anything like i'd expect from a professional organisation.

    I think they've proved that it can work, but not that it does work.

  • Comment number 28.

    I asked the following

    1. Is there a God?
    No commital, however it told me it would get back to me if I entered my email address.
    2. Are the Knights Templars the keepers of the Holy Grail.
    3. What is Love?
    4. Can rabbits sew?
    5. What number am I thinking of?
    6. What is an elbow?
    ALL FAIL
    7. What is artificial intelligence?
    Apparently it's a film by Steven Spielberg.
    8. Who is Wayne Rooney
    A "soccer" player from Liverpool. It even gave me a timeline of his life starting in 1900 (he doesn't appear until 1985)

    Good stuff - bored - wandered off.

  • Comment number 29.

    Hmm.

    I tried a fairly generic query for "web engineering", my own research area and not particularly esoteric. Not a thing.

    But it did manage a power series expansion for pi, which was nice

  • Comment number 30.

    Does Wolfram work?
    Short answer : No.

    I asked:
    > Is wolfram better than Google?
    > Wolfram doesn't know how to handle your input.

    > Are you better than google?
    > Wolfram doesn't know how to handle your input.

    > How to manufacture LSD?
    > Wolfram doesn't know how to handle your input.

    > Do you know anything?
    > (After a long pause) Wolfram has exceeded it's current maximum load.

    Oops, I think I killed it.

  • Comment number 31.

    In a word, NO.

    It thinks there are less than 6,000 people living in Wales, but more than 117,000 living in one of it's cities.

    http://www32.wolframalpha.com/input/?i=newport+wales

  • Comment number 32.

    It doesn't work, at least not yet. As someone suggested, its more like a proof of concept.

    I asked it "How many lawyers in Israel". It couldn't cope with that.

    OK, maybe its still US biased. "How many lawyers in California". It couldn't cope with that.

    Then I gave it "Alexander Ryback", the gentleman who just won the Eurovision song contest. It couldn't cope with that.

    I tried one more, more scientific question. "Common vagina disease". It couldn't cope with that either.

    Long way to go guys, long way to go.

  • Comment number 33.

    It is EXCEPTIONALLY weak in many many places.
    Of course it's great with some stats, yes you can do calculations (basic ones that us normal souls want to do have been on Google for years) and theres some really good tricks but this is in no way a search engine, and in no way a Google killer.

    It gives sources but in a mix rather than applying the sources to the individual answers (as you might get several when it doesn't know what you want) so you have no way of assessing it's value, just as we do with links on Google.

    Weather is a good example. Ask for the temperature in Cambridge and it gives a figure that is over 2 hours old and a couple of extra figures for locations over 20 miles away. If I go into Google and search for Cambridge Weather I find a nearly live figure for a university system that is also closer.

    Ask 'who is batman' or any other fictional character and it'll automatically pull the information from IMDB and Wikipedia. But if I went to Wikipedia you get masses of links to every permutation of the Batman history.

    One example I saw shouting how great it was gave the example of calories of a doughnut. Amazing! But what sort of doughnut? Where was it bought from? What flavour? So I tried 'calories in a dunkin donut'. Didn't understand my question. Put that into Google and first link was to the Dunkin Donut nutritional information on the companies website.

    It doesn't even give links for further information! I'm supposed to just trust them? A lot of their sources seem very difficult. I also can't see anyone accepting WolframAlpha - Primary source as an acceptable reference and if you can't seperate references out (as in normal scientific papers) what good is it?

    I like it, it's very clever, but it's a massive letdown from how they sold it.

  • Comment number 34.

    Sadly no....well not for non US based searches. Why do all these companies assume that the world is made up of the US only.

    Maybe it will get better but I'm surprised considering where Dr Wolfram comes from!

  • Comment number 35.

    You're all asking the wrong questions - try 'why did the chicken cross the road?' and you get the correct answer. I guess Wolfram does have a British sense of humour after all ;)

  • Comment number 36.

    Type in how many people live in Scotland and you get an answer 1699!

    What's all the fuss in any case, the START system at MIT have had a better system for years.

  • Comment number 37.

    Ultimately, I think Wolfram Alpha will fail for 3 reasons:

    1) It feels geeky. That alone will prevent it from being adopted by the mainstream. Everything from the name to the look and feel, not to mention it's sciency intentions could limit its audience to mathematicians, scientists and students.

    2) Can users trust it? The 'knowledge' comes from its own servers and data sources, with little or no indication of source. Whereas from the internet, the user gets the perception of control: you can find out for yourself which websites look trustworthy and which search results don't and make your own judgement based on the common answer given by many. Wolfram Alpha gives one answer and one answer only - its own answer - with nothing to verify by. And can you really trust a computer to interpret a wordy version of a mathematical formula correctly every time?

    3) Limited scope. Partly related to point 1, it is great as a fact-finding tool for certain pub quiz or trivial persuit questions (at least the ones in the geography or science categories) but next to useless when trying to find out news, opinion, anything entertainment related and the like. And Wikipedia, for example, does all these things already for no more amount of hassle.

  • Comment number 38.

    Oh, I thought of one more reason it will fail:

    4) Google has become a verb, the very meaning to many people of an internet search. "Google it" has become synonymous with search, just as 'iPod' has become synonymous with 'music player'. Therefore, nothing else matters, however good rival products are, and that includes Wolfram Alpha. And once Google Squared is finished, I'm afraid to say that Wolfram Alpha will be finished too.

  • Comment number 39.

    So far, pretty poor. Its almost clever. Certainly gets some answers... but there are huge pieces of data missing.
    You try putting in a search for UK Earthquakes... and then filter it down to 30 years - Greater than a magnitude of 2...

    Absolutely nothing over the last 30 years.
    Try putting in the name of a town like Wednesbury, or its council area Sandwell and you get nothing.
    No Burnham-On-Sea... It thinks Wales is a town... I asked it for the 100th Sunday of the 21st Century... and it managed to compute the question correctly, but fail to provide an answer.

    I look forward to Wolfram Beta... as Alpha leaves a lot to be desired.

  • Comment number 40.

    Well it succeeds at least in doing what the dictionary has done for school kids. Let's you look up swear words. Had some fun with that.

    It also knows the average airspeed velocity of an unladen swallow, so long as it's a European one.

  • Comment number 41.

    When I asked for the "answer to the ultimate question", it did indeed come up with the correct answer of 42 (according to Douglas Adams).

    However, it was not able to tell me what the question actually was.
    So I guess it really is just a work-in-progress.

    Sigh.

  • Comment number 42.

    I think Wolfram might have more success if they they tried to build a working version of the "Total-Perspective Vortex" (also described by Douglas Adams).

  • Comment number 43.

    Dire. I asked it a relatively straightforward programming question

    How do you convert to upper case in java

    the answer?

    JAVA

  • Comment number 44.

    Quality!
    Q: What is the meaning of life?
    A: 42
    Q: How long is a bit of string?
    A: Computation timed out.

    Most of my other childish questions failed......

 

The BBC is not responsible for the content of external internet sites

BBC.co.uk