New BBC Site Search
This week, we launched our new site search product. This launch is a big step on our journey to deliver a single, unified and consistent discovery experience for the BBC.
Try it out with the search box in the top right-hand corner of most pages (or, if you can't think of something to search for, try Liz Hurley, Miranda, Unemployment, Spain). Let us know your thoughts using the form at the bottom of this page.
The new search is based on the Search+ platform, which I blogged about earlier in the year, and includes many refinements and enhancements.
This new search supersedes both the standard search product and the three-column Search+ product to offer a single site search experience for BBC users.
Before I go through the new features in detail, I'd like first to give you an insight into our product vision and discuss some of the issues that have shaped it.
To scope or not to scope
When I started in this job, there were hundreds of distinct sites on bbc.co.uk and most of those websites would have their own search "scoped" to their content. Not only was this inefficient and wasteful but also it led to an inconsistent experience in which search boxes in exactly the same position on, say, the Schools website and the History website would produce different results. People were often confused; indeed, I remember analysing the logs for the Gardening website search and discovering that the top result was for [eastenders] and returned no results.
We decided to reduce the number of scoped searches and make the search box in the top right-hand corner of every page return the same results, wherever you were on the site. However, this was not universally popular. People tend to think of bbc.co.uk as only that part of the website that they like to use. Many people only use the News website, or GCSE Bitesize or iPlayer and, not unnaturally, are bemused when they search for something and get results from another area. We get many emails asking for the search box on News to return results from News only. There is even an online campaign dedicated to achieving this goal!
So, on the one hand, people are finding no matching results in scoped searches, even though relevant content exists elsewhere on the website. On the other hand, people like the convenience of scoped searches, particularly when they only use one of the BBC's services.
The table below further illustrates this problem. It is a log of the top queries coming from different BBC websites on one day in June 2010. The three columns show where the searches originated (on the iPlayer, News and Sport websites). Those queries in black are the ones for which the best results will come from that website (so, for a search in iPlayer for [luther], we should expect the best content to be from iPlayer). The red ones are the queries for which the best results will come from other parts of the BBC website (so, for a search in iPlayer for [revision science gcse] we would expect the best results to come from Learning rather than iPlayer).
So, as you can see, the picture is pretty mixed, with a closer correlation visible the higher up the search logs one looks between the originating domain of the search and the provenance of the optimal result set.
Clearly, we need to have a search product which recognises that many users want to scope their searches to the domain they are currently in while simultaneously revealing great, relevant content that happens to be in another part of the website.
With this in mind, we created the following Product Vision.
Our aim for search is:
1. To create a scoped search for each BBC product (News, iPlayer, CBBC and so on) that "understands" the domain knowledge behind each product and uses that information to create the best possible sorting rules, display and filtering options within search. In this way, we will provide the richest search experience available anywhere on the web for each of our products.
2. To allow users to search across product scopes to see everything that the BBC has on a subject.
3. To ensure that, regardless of which scoped search someone uses, they are always exposed to the best relevant content the BBC has to offer. This will ensure that, for example, a heavy News user gets to know about iPlayer content relating to their query.
4. To offer related results from non-BBC sources.
5. To make it possible for people to discover trending topics on the BBC site and make feeds of results available to be embedded in other products (on and off the site). To allow search result pages to be recommended from within other BBC products.
We think Search v2 starts to fulfil a number of these ambitions. Let me run through some of the new features and demonstrate how they relate back to the Product Vision.
1: A scoped search for each BBC product
This example is a search for [graeme swann]. The results have been scoped to Sport. This shows the main elements of the page design, which I have highlighted with red numerals.
1. The central column will contain the main list of results. The large heading (in this case, Sport) shows that you are searching within the Sport scope.
2. The left-hand column will contain navigation to the other scopes and filtering options such as media type, date range and sorting choices.
3. The right-hand column will contain search results from other parts of the BBC (Elsewhere on the BBC) or other websites (Elsewhere on the web).
The styling of the new search results uses the BBC's Global Experience Language, which brings it in line with the rest of the website and which Bronwyn comprehensively explains here.
In the image above, you can see four sets of filters in the left-hand column. The first is the category filter. It shows you how many results there are for your query within each category. Below this are "scope-specific" filters. For the Sport scope they are:
1. Allows you to filter your results to show only video, audio or text & pictures. The default setting is All Media.
2. Allows you to set the earliest and latest dates from which you want to find results.
3. Allows you to change the ordering of your results. As I said above, the default order is Newest First, but you can also choose to order by Most Relevant First or Oldest First.
Over time, all scopes will be given filters best suited to that particular type of content, so Learning, for example, will have filters such as Subject, Syllabus and so on.
Context-driven search results
As discussed above, there is a strong correlation between the part of the website that a user is exploring and the type of content they are searching for.
So, we have set the system to return results automatically from the category that relates to the section of the website you are using - if you are on the News site when you enter a search term, you will get results from the News category in the main, central column. Likewise, if you search from Sport, you'll get sports results.
Over time, we will extend this functionality to other sections of the site - iPlayer, Learning, Food and so on. The eventual aim is that all products on the site will scope to their own search category while staying within a single, consistent design. As this happens, we will remove the old "scoped searches" like this one.
From today, however, if you search from any part of the site other than /news or /sport, you will get combined results on our All Results page. You will know which area you are about to search in from the text inside the search box. On News pages the box looks like this:
On pages where you are searching All Results, the box looks like this:
2. Allow views across BBC products
When you hit the All Results page, the central column will contain the top results from all the matching categories. The categories are ordered by their relevance to the search query, using an adaptation of the Smart Zones process that I discussed in an earlier post. Here is the [graeme swann] query from above, run on the All Results scope.
Note that the page layout is the same as in the earlier example. However, because I haven't scoped my search, I will see results from all areas of the site in the central column. In this case, Sport is rightly deemed most relevant to the cricketer, followed by Blogs, News and then TV & Radio sites. A maximum of three results are shown from each category.
Likewise, the query [christmas cake] returns Food as the top category. (Ignore the Editor's Choice result in each of the next three examples - I deal with this feature in detail later on.)
Whereas a search for [iceland] returns News as the top category.
And a search for [trigonometry] will put Learning content at the top.
You can get to all results for a particular category by clicking the links in the category title (or in the navigation in the left-hand column).
Many search result pages will also include a selection of editorially chosen results that we believe are most likely to be what you are looking for. If you search for a programme title, say [miranda], you will see a link to the programme website.
If you search for a country, say [burma], we will highlight the BBC Country Profile (as well as other editorially selected features) and if you search for a British place, say [blackpool], you'll get a link to the nearest BBC Local website.
3: Ensure that related content is never hidden
Elsewhere on the BBC
As I said in the Product Vision, our aim is never to hide great, relevant content from users, regardless of the scope within which they are searching. So, whenever you search inside a scope, you will always see the Elsewhere on the BBC module showing results from other parts of the website.
4: Offer related results from non-BBC sources
Elsewhere on the web
Below the Elsewhere on the BBC module, you will see the Elsewhere on the web module showing related results from around the web. These are a mixture of editorially selected websites and algorithmically driven results.
5: Linking to search results
Linking to search results from within the BBC
In the example below, you can see that iPlayer now links to BBC search. In this case, a programme about Operation Mincemeat links to search result pages for World War II, Adolf Hitler and the show's presenter, Ben Macintyre. This is possible because search produces a controlled vocabulary of around 40,000 topics that are available as links and which I wrote about previously here.
These 40,000 topics are also available as RSS feeds. Click on the orange symbol in your browser's address bar to get the feed.
How we put this together
For those with a technical inclination, our senior engineer on this project, John Muth, will blog on the technical approach next week and I will add a link from this blog then.
Finally, a bit of news about the BBC's search team. We are moving to Salford in the summer as part of a BBC-wide relocation. As you can imagine, such a big change has resulted in a fair amount of turnover within the team and so we are now actively hiring software engineers (predominantly Java) and other roles, including my job - Technical Product Manager, Search. You can read about the recruitment process here.