Text question and answer companies
In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions
Text-based question and answer services are the latest way to look clever. For about £1 you can text a question and a few minutes later receive an answer. But we've heard that some of the answers sent by some of the biggest question and answer services are wrong.
Watchdog looked at three big names in the market: 118 118, which is the market leader; Texperts, which is now owned by 118 118; and AQA 63336. The AQA stands for 'Any Question Answered'.
Ben Archer is a keen rugby fan, and relies on texting 118 118 to find out local rugby results because he doesn't have access to the internet over the weekend. But when he texts them asking about his team in Bury St Edmunds, Suffolk, they text him answers about a team in Bury, Lancashire.
He says he realises no one's perfect, but that if 118 118 is providing a service it should be accurate.
Watchdog experiment
To test the companies' accuracy, Watchdog conducted an experiment last week. We texted 20 general knowledge questions to 118 118, Texperts and AQA 63336. We sent the same questions twice in case the companies replied with an incorrect answer the first time. So 40 questions were asked in total.
The results showed that Texperts sent us the correct answer to the 40 questions only 65 per cent of the time. On some occasions it told us it didn't have enough information to answer our query. Once, the answer it gave seemed to be for a completely different question we hadn't asked.
118 118 correctly answered the 40 questions 72.5 per cent of the time. It sometimes answered one question with a wrong answer, but when we texted the company again it sent us the right answer. On one occasion it gave us an answer that we'd made clear in our question we didn't want.
AQA 63336 were the most accurate of the three companies, answering 92.5 per cent of the 40 questions correctly.
When 118 118 and Texperts can't give you an answer they won't charge you. AQA63336 will give you a refund. If you can prove the answer they've sent you is wrong, all three will give you your money back.
Celebrities versus the companies
We tested three clever celebrities against the answers we'd been sent by the companies. We assembled a crack team comprising quiz maestro Keith 'Cheggers' Chegwin, Cambridge graduate Konnie Huq and all-round know-it-all Iain Lee. We asked them the same questions, though we only asked them once rather than twice. They also had access to the internet, as the text companies do.
Our celebrities didn't beat AQA 63336, but they did answer 18 out of 20 questions correctly, giving them a score of 90 per cent. Better than both Texperts and 118 118.
AQA 63336's response
We've 1,000 mostly UK based-researchers who together answer an average of 15,000 questions a day, and have seen peaks of up to 25,000 questions a day. Since launching AQA 63336, the world's first text Q&A service on a premium text code, we've answered over 17 million questions from 2 million customers.
As a company we always try to provide great customer service, and as a company our top goal has always been to provide a high quality service. All questions texted to AQA 63336 are answered by human researchers, who strive to provide not only the right answers, but answers written in the Queen's English in a way that's warm, positive, and often witty. By being a human-based service we provide great value to our customers, but we've always recognised that because it's a human-based service, we'll make mistakes. There are, of course, some questions where there is no right answer, or answers are disputed. However, if we do get the answer wrong, we will always apologise, refund the customer, and importantly correct the information that's been given out. As a note, we currently only get one customer complaint per 10,000 answers sent out.
As we've said, all questions texted to AQA 63336 are answered by human researchers, who strive to provide brilliant answers - answers that are fast, pithy and accurate, but always fill up a text message, and often provide additional information that will wow our customers. There are occasions where it's possible to have more than one right answer to a question, so sometimes we do give different answers. As mentioned earlier we are a human service so also we can occasionally get it wrong.
Response from 118 118 and Texperts
Due to the recruitment of new staff, less experienced employees have made some mistakes. This is being addressed by further training and supervision.
Our own internal quality control shows that we're achieving more than 90 per cent accuracy. The vast majority of questions carry no ambiguity. The clever questions posed by the BBC simply do not represent the actual range of straightforward questions received by 118 118 from the public.
We acknowledge all questions but do not reply to certain questions that incite defamation, racism or other forms of discrimination, or for information that might constitute civil liberty issues, unwarranted invasion of privacy, or for information such as medical or financial advice that requires specialised, qualified advice.
In the event we're quite unable to answer a question, we do not charge for our response. Note that this is different to AQA, where the user is automatically charged for asking the question, regardless of response. We do not charge on the inbound question.
We're very careful to only advertise Ask US Anything, and not Any Question Answered, because we recognise not all questions can be answered. We believe we are being wholly honest in our claims and up front about what we can and cannot do. We further believe the sweeping advertised claim of AQA, Any Question Answered, would require a score of 100 per cent to meet expectations.
We're proud of our new service, which has become the market leader very quickly. We accept there have been some teething troubles, but believe the service is delivering an important service to the British public, and look forward to ever higher levels of accuracy and usefulness.
© MMIX~RS~q~RS~~RS~z~RS~14~RS~)
Comments
Sign in or register to comment.
This comment was removed because the moderators found it broke the House Rules.
People should save there £1 for an answer and use GPRS/3G, for a quick google search it wont cost more than a few pence. Most contracts/payg phones have inclusive data usage so generally it wont cost a penny!!
Complain about this comment
This comment was removed because the moderators found it broke the House Rules.
Recently when asked "What height above sea level is Leicester City centre" the given response by 118118 was "We only found that the town of Leicester rests among the mountains fourteen hundred feet above sea level." Either that facts are off or I am severly lacking in observational skills
Complain about this comment
Texperts no longer exists. It was bought by kgb (the people behind 118118) and the 66000 shortcode diverted to 118118. All former Texperts have been laid off.
Complain about this comment
It's a shame that Watchdog appear to have missed the most crucial piece of information about this test. In December 2008, Texperts were taken over by kgb (the owners of 118). As such, all questions sent to Texperts were automatically sent to staff employed by 118. As part of the takeover, Texperts didn't answer customer questions any more.
It shows that on two occasions 118 have been unable to answer the same question. It's a shame the better service, AQA, are unable to compete with 118 on advertising. If this test was actually taken when all three companies existed the results would well and truly prove how bad a service 118 actually is.
Complain about this comment
I just started to defend 118 saying how they saved me a cogestion charge fine last year. It was then pointed out to me that Tower Bridge is outside the congestion charge. I've just checked the Cogestion charge web site and it's true! 118, you owe me £8.00.
Complain about this comment
This comment was removed because the moderators found it broke the House Rules.
The response from Texperts is very petty. There's no big semantic difference between "Ask us any question" and "Any question answered". Both imply the promise of a correct answer.
[Full disclosure: I work for AQA.]
Complain about this comment
Most texts are answered outside of the UK in Manila and other countries. They will always make mistakes due to their lack of local knowledge.
True they have new recruits, they have replaced a large number of UK based staff in Cardiff.
Complain about this comment
When i saw this on watchdog earlier it certainly rang a bell with me.
I used 118118 during the week to find out what time my train would arrive in at Watford Junction as i had to change there for another train i thought i would try 118118.
I entered all my details (train times/ stations etc) and i got a text back saying the train i was on was a direct service when i knew it wasn't and i knew it WOULD be stopping at Watford Junction. I text them back saying it was defiantly stopping at Watford Junction and got a reply saying no it wasn't it was a direct service to Euston. Low and behold my train did stop at Watford Junction and i made my 2ND train in time.
I have used AQA many times before and mostly have got very good answers from them so i will be sticking with them from now on.
Complain about this comment
This comment was removed because the moderators found it broke the House Rules.
This comment was removed because the moderators found it broke the House Rules.
Is there any chance that the BBC could publish the list of questions asked, along with the answers given by all three companies? It'd make interesting reading, as from my observations of 118118, irrespective of the accuracy of answers, their spelling and grammar is frequently appalling.
A quick glance at their live feed can confirm their car-crash grammar. I warn you, you will be drawn into the vortex once you start watching it!
Complain about this comment
I work for AQA too - just so you know!
Complain about this comment
What about Public Libraries? There is a national service http:www.askalibrarian.org.uk that is available on smartphones etc. It isn't a text service, but it is staffed by people whose job it is to find information. Staffed by the UK Mon-Sat 9-5 and the US after that, we don't profess to know all the answers all of the time...but it is 'free'.
Complain about this comment
Thanks to the Watchdog team for this interesting piece, testing the quality of text-based question and answer services. Obviously SMS is a very convenient medium for folks with a mobile phone. However, as more of us get smart phones with mobile internet access, this opens up other means of communication such as instant messaging and chat. Perhaps, rather than paying £1 per question to commercial services, people would like to try instant messaging/chatting with a librarian for free via the Enquire ask a librarian service http://www.askalibrarian.org.uk/ This is a 24 hour, live question answering and enquiry service offered by public librarians across England and Scotland in collaboration with partners in the United States. Librarians love helping people find the information they need. And what's more, they're professionally trained to do it!
Complain about this comment
It seems the programme worked & 118 have now done away with the ghost of a service that was texperts. A sad loss to all those involved. It was never as bad as you showed it when it used real, intelligent agents.
Complain about this comment
This comment has been referred to the moderators. Explain.
This comment was removed because the moderators found it broke the House Rules.
I've texted 118 118 a few times. On occaision it has got it right. I once asked for some instructions for a game, and it referred me to a website. Had I had the internet, would I have not looked this up myself?! I replied ssaying this and they did get me the correct information. Another time I asked for a beauty salon in the centre of the town I live in. I specifically said 'centre' yet I was sent the details of one a good 6 miles away. Great, thanks.
Complain about this comment
View these comments in RSS