The new search engine has an increased spelling tolerance
Hi I’m John Barratt, Senior Product Manager for BBC Search.
I'm delighted to be able to tell you about a new underlying search engine and also a responsive search results page design, with a look and feel optimised for mobile, tablet and desktop screens, both of which went live yesterday.
This represents only our third change of search engine technology in the 20 years that BBC Online has been in existence, and the most important update in the past 4 years.
The new design brings search in line with the visual design of recent Sport and News releases, and presents results more clearly. We hope that the redesign will encourage more users to explore more results presented to them.
We’ve also introduced several features that will improve your search experience:
A new ‘All’ results page
By default, the new search engine displays results from all across the BBC, rather than listing results from a specific area of the site (e.g Sport, or News). This allows a user to navigate quickly to the most relevant result for their search term.
New relevance algorithm
The new search engine ranks the relevance of a document based on how often your search term appears in a document's title or headline, summary and then the body of the document itself, in that order. Then, we ‘boost’ the results depending on when they were published or broadcast. So if there were 2 news stories on Syria the one that was published 20 minutes ago would be ranked higher than the one published 2 days ago. But this upweighting only significantly impacts articles published in the last 24 hours.
Increased Spelling Tolerance
Next time you make a slip on a keyboard the search engine will guess what you meant
Often when I’m typing into a document or a search engine I make a spelling mistake. It’s easily done, especially on touch screen devices. We have seen over 200 different ways people have spelt 'Eastenders' when searching for that programme. The old search engine needed to be told exactly what a misspelt word actually related to otherwise it gave you no results. The new search engine tries to work out what word you meant to search for and return results that best fit. So next time you make a slip on a keyboard and accidently search for ‘Wastenders’ or ‘Ucraine’ our search engine's probably guessed what you meant.
So why have we made these changes?
Well the new search engine replaces a system that’s around 12 years old. Due to its age we’ve seen issues around its performance with queries taking anywhere up to 4.5 seconds to be processed and returned to the user. In search engines terms that's a lifetime. The new search engine returns results on average in around 0.5 seconds (depending on your connection speed).
Too many people become so frustrated that they leave bbc.co.uk via the search pages - up to 15% of our users. And of those who perform a search, just 29% actually click on a result. We're determined to improve both of these key measures, and believe that our more relevant and helpful results will ensure that more people click on more of our top results.
Finally some of you may have at one time or another completed an online questionnaire asking what you think of search and giving it a score out of 10. Search has one of the lowest scores of any of the areas on BBC Online. Your comments reveal that you want an experience similar to search engines such as Google and Bing and that your main frustration is not being able to find the content you are looking for. We're confident that the changes we're making today will markedly improve your experience of the service.
We have also removed a couple of features:
The old search used to have a series of filters that were originally released to enable users to refine their search based upon date, relevance or media type. However they only ever appeared fully on news and sport categories and partially on the blogs category.
Also they weren’t used very often. Between 0.02% and 0.6% of searches were 'filtered'. So we’ve taken the decision to remove them on the basis that they weren’t across all results, were infrequently used, and the new algorithm returns results in an order that doesn’t require them to be filtered.
When you searched from News and Sport pages you used to only get results from those areas. The new search gives you ‘All Results’ for a keyword, which you can then refine to just news or sport by using the subject filters along the top of the screen. By presenting a set of results from across the BBC we give people more options and will encourage more people to click on results.
We know that we won’t have got everything right first time, and that changing the way things look can be confusing at first. So, we will monitor your feedback on this blog, via the online quality survey and the usage activity we continually log for the service. This platform and the new design actually allow us to respond to user feedback more quickly by being able to change elements both on the page and in the back end more easily than ever before.
Finally, this isn’t the end (but to paraphrase a great quote) but ‘the end of the beginning’. You will see more changes over the next 6-12 months as we continue to refine the design and the service as a whole.
John Barratt is Senior Product Manager, BBC Search, BBC Future Media