Earlier this week we began a phased roll-out of our new site search. New features include:
- A brand-new search which intelligently tailors the display to the specific query the user enters
- The addition of featured content to search result pages
- Improved links to non-BBC content
- More accurate results through improved meta-data creation
Our aim in designing this new search experience has been to present search results in a way that makes sense of the huge variety of content available on bbc.co.uk
. It should be effortless to find a specific piece of content and enjoyable to explore everything that we have on a subject.
The richness and diversity of the BBC's internet content (News, Blogs, iPlayer, Weather, Sport, Recipes and so on) places demands on site search that are different from more focused websites and this led us to explore original solutions to enterprise search.
As you will see in this post, our solution is a major departure from traditional enterprise search designs. So, for the rest of this post, I'd like to give you some insight into how we reached our decision to make such a significant change to the way we deliver our search results.
As always we would love to hear your views on these changes.
The challenge of site search
Last year, Erik Huggers challenged our team to "deliver a step-change" in the quality of site search.
This was a daunting challenge, but as always the first thing to do was to define the problem. This meant analysing what users are searching for and which results they are clicking on.
To explain what we found, consider this list of the 50 most popular searches in December 2009.
The third column shows the click-through rate (%CTR) for each term, i.e. what percentage of people who searched for that term actually decided to click on a result. The higher the %CTR the more satisfied we believe users to be.
Notice that I have colour coded the different search terms. Green are searches where the user is looking for something specific: usually a part of the BBC website (iPlayer, the Weather website, Bitesize) or a named programme (EastEnders, Top Gear, Doctor Who). Red ones are searches where the user is looking for something about a topic - a particular person, a country or city, a subject or news event.
Look what happens when you average the %CTR for these two types of search:
Looking for something specific 83.86%
Looking for something about a topic 63.91%
As you can see when you want to get something specific, the BBC site search is pretty effective. But if you want to find things about a person, place, subject or event the experience is patchy.
So we began to re-develop site search to improve this type of search, which I call Topic Searches.
To start with we asked "why are Topic Searches less successful?" The answer is surely that the intentions of the person doing this type of search are less clear. When you search for merlin or top gear or travel news the intended results are easy to predict. But when you search for iran, what do you want? The latest news about Iran? Some background information on the country? A programme that's available on iPlayer?
What about searches for Delia Smith? Do you want to know when her next programme is being broadcast? Do you want to follow one of her recipes?
Results display: the problem with search result lists
We then looked at how the results were being presented and whether this was hindering users' understanding and therefore depressing the click through rate.
The traditional way to present search results is as a single list ordered by relevance to the search term. This has been our approach up until now. But it has never been great for topic searches. Here is what one interviewee said during some research we carried out:
"The search just throws everything at you, you would have to spend ages just looking through the pages to find what you need."
There are two big problems that make a single list approach very difficult:
- You need to understand what the user means to know what relevance scores to give to their results. Two searchers may want two different things but use the same search term - one user may search for swine flu looking for a description of the symptoms whereas another may use exactly the same search term but be interested in news about the latest outbreaks.
- Different types of content are relevant for different reasons. The most relevant news article is the most recent one (unless of course someone is looking for a specific article). Whereas for archive programmes or background articles date is much less important, the most relevant thing is the one "most about" the subject.
This means that when you try to produce a homogenised, "relevance ranked" list of results you need to add biases and boosts to different types of content to get the right results to the top of the list. With the diverse content on offer on bbc.co.uk, this is nearly impossible.
(It's worth noting that the major search engines have also been moving from a single list view to a more modular set of results over the last few years. Google for instance include modules of results from their verticals - News, Video, Blogs, Maps and now Tweets - blended into their Web results. Google's Marissa Mayer laid out their strategy in a post in 2007, Danny Sullivan has a great explanation too.)
So our solution rejects the single list approach and instead introduces "Smart Zones".
The New, Enhanced BBC Site Search: Smart Zones
This is a close-up taken from the results page I got when I searched the new system for David Cameron
As you can see, the results are split up into sections or "zones" - in this case: News, iPlayer and Knowledge (the zone which holds background and in-depth content). The content in each zone is ordered in the most appropriate way - so that you see the latest News but the most relevant Knowledge items.
Of course zones are nothing new in search result pages but - this is the clever bit - these zones order themselves on the page depending on the search query, the matching content available and what we think users find most important. So every time you search you will see the best possible ordering of available results.
For example, compare the results above with a search for David Cameron I did on Sunday morning just after he had been interviewed on The Andrew Marr Show. You can see that the News and iPlayer zones have swapped places as the system has calculated that the interview is likely to be the most popular result.
When I searched for Ryan Giggs, the Sport zone appears in the results.
Searchers can see all zones or focus in on one particular zone. Here I have searched for Jonathan Ross and I have selected to expand the News zone to see more News content (note all the other zones are still available on the right of the screen reminding you that other relevant content is available).
Other Enhancements to Search: Highlighted Content
This is a search for George Orwell
. As well as search results we also show profiles for certain people. In this case it is from the BBC History site
Likewise, a search for India returns the BBC Country Profile and the latest weather forecast. (Note that no new content is being produced here, we are just finding new ways to draw attention to our best web pages). We will be working to increase the amount of highlighted BBC content over the coming months and highlighting the best content from other websites.
Other Enhancements to Search: Extra results from around the web
Here I have highlighted another zone that appears for most searches - Around the Web
. This shows relevant content from other news providers. In the image below the search was for Simon Cowell
Other Enhancements to Search: Extended query terms
Here is a close-up of a section of the results for a search for Prince Charles
. Some journalists will refer to the Prince as Prince Charles, others as The Prince of Wales. Our system knows this and automatically includes synonyms of your search terms (as shown in the detail below).
Other Enhancements to Search: Better results through added content structure
One of the advantages that site searches have over internet web search engines is that we can influence the way content is produced to improve the quality of our search results. We have put a system in place that uses search to suggest the most relevant tags for BBC content. Content producers accept or reject these tags before they are committed to the system. The tags are then used to influence the results that are returned when you search the website.
Note this system is in its infancy so it will take some time before the improvements in result relevance are apparent for every search.
Other Enhancements to Search: Addressable search results
All search result pages have unique and persistent urls. This makes it very easy for people to link to everything that the BBC has about a topic. In time public APIs will allow other systems to request feeds of content about a topic.
The roll-out will be phased. We'll be monitoring our metrics (usage, click-throughs etc) and the feedback we receive to tune the product during the deployment cycle.
At the moment only about 11,000 search terms will return the enhanced results detailed above.
Over the coming weeks more and more searches will return the enhancements until every search works in this way. There will be additional zones added into the results and we have a slate of new features that are in development.
We will also be using the technologies we have built here to produce exciting new applications. One of the objectives we are focusing on is how we can use these tools to make you aware that there is new content about things that are of interest to you as soon as it is published. Watch out for something special when the BBC Homepage re-launches in the very near future.
Matthew McDonnell is Head of Search, BBC.