A new way of navigating the BBC’s archive of permanently available programmes
Business Analyst, BBC Radio & Music Multiplatform
Currently there are over 15,000 permanently available programmes, largely radio programmes, on the BBC website dating back decades, but they can be very difficult to find. From today, we’re launching a new piece of technology called ADA (Automated Data Architecture) that unearths and helps people navigate the BBC’s rich archive of permanently available programmes.
As you can see below, it adds a list of related topic tags under the description of the programme. So if you’ve just listened to an episode on Ada Lovelace and were interested in other notable women of the Victorian era, you can now click that tag and find all the permanently available programmes on that topic. There are programmes on Beatrix Potter, Florence Nightingale and Sylvia Pankhurst to name a few. There will also be up to three recommended programmes on the right hand side, with a link to the topic that connects them.
This seemingly small change to a programme page can lead you down interesting little alleyways to fascinating places you never expected to visit. For example, starting off at Ada Lovelace can take you all the way to a programme on Julius Caesar via ‘the Byron family’ followed by ‘Fellows of the Royal Society’ then ‘Captain James Cook’ and finally the ‘Deaths by stabbing’ topic tags. Give it a try here and see where you end up.
Some programmes like Desert Island Discs, which have a lot of programmes available, have navigation which is tailored very carefully to the brand. This makes it easy to find programmes but also means the system cannot be re-used across other BBC brands or programmes.
Most programmes do not have any way of browsing by subject though, especially one off documentaries and short series. They can be found by searching, but only if you already know what you are looking for and what it is called. Even then it’s very hard to find something new or surprising, even though there are so many programmes available.
Some attempts have been made to bring these hidden gems to light, but these have always involved a lot of manual curation of content by editors to find and keep the collections updated, which simply isn’t sustainable.
What we wanted to do was to find a way of connecting up all of these programmes in an automated way, and make it easier for our producers so they could spend all their time on creating the programmes we love. The BBC News and Sport websites use linked data to populate their pages, so with the infrastructure already set up to create and add tags, it seemed like this would be useful in connecting programmes.
However, it’s not as simple as just adding tags; you need to provide journeys between the tagged things. In Sport they have a structure which describes the relationship between things, which is used to power their website, so they know that a person belongs to a team which is part of a league which is part of a sport, therefore they only need to tag a story with a person and it will pop up on all of the relevant pages on the sport site.
In programmes there is no clear structure that can be added to the tags to create links between programmes, and with the subject of programmes being as diverse as the A470, Munch's “The Scream”, the cronut, Canada geese, existentialism and the Battle of Bosworth Field, it was clear that creating our own would be an enormous and maybe even impossible task.
We did look at using a variety of existing systems and approaches, such as Dewey Decimal found in libraries, and crowdsourcing where enthusiasts can help tag programmes, but none quite fitted the bill. After a great deal of searching and trial and error we found that the categories in Wikipedia looked very promising.
Categories are added by Wikipedians because they believe them to be important facets of the subject of the page, so they are likely to be more interesting than a dry statement of fact about each subject. They are also checked by other Wikipedians and will be removed if they disagree, so there’s a certain amount of quality control built in as well.
Using these categories, we built a beta system where we simply tagged each programme with its subject, gathered the Wikipedia categories for that subject automatically and matched them up, providing links between the programmes. This automatically created the navigation architecture that we needed, without us having to spend hours designing and updating it.
We needed to test it and put it through its paces so we looked at In Our Time, which has a rich archive of over 700 programmes with a wide range of subjects including people, places, events and philosophical terms. We felt if it worked for In Our Time it would work with anything.
The beta was launched in spring of last year and we promoted it on BBC Taster so we could get as much feedback as possible. We got a great response, with an overall rating of 4.15 stars (out of 5) which was really encouraging. Even better though, was the feedback we got through the beta itself, where people could actually tell us what they thought. This was brilliant because, given that most of them were positive about the experience and we were looking at rolling it out across all programmes, we needed to know what people liked and didn’t like so we could improve the whole experience.
Some people preferred navigating topics in other ways, such as by date or A-Z, and these will still be available in the programme pages for those who prefer that, alongside the additional topic journeys. Some thought a hierarchical structure for the topics would be better, so they could look at a broader subject and refine such as choosing ‘history’ then ‘19th century’. The difficulty with that sort of navigation is that it requires a lot of man hours inputting things into categories and also removes much of the serendipity of finding new links between things, which people told us they really liked.
Interestingly, some people realised that manually tagging all of these topics would be incredibly time-consuming, assumed that’s what we’d done and felt we shouldn’t have spent so much time doing it. But of course that was exactly the problem ADA was solving. And another user commented that they found they were learning about the subjects of the programmes just through the journeys and the tags, even before they had listened to them.
Many people commented on the fact that once they’d found new programmes they couldn’t download them in our stripped-down beta. This won’t be a problem in the new version as it will be integrated with the existing programme pages. Others asked for things we hadn’t thought of, like the ability to subscribe to the podcasts by topic and to favourite topics so they could be updated with new programmes of interest automatically. They seem like great ideas, and although we’re still working on perfecting this, they’re definitely on our list to look at when we’re done.
Lots more asked whether we could roll this out to other programmes, which is what we will be doing, so you should be able to find everything on social philosophy, for example, not just for one programme but across a variety of them.
We’re starting out small with a handful of programmes, including In Our Time of course, and will continue to expand over the coming months. We really hope that you enjoy finding new programmes and discovering more hidden gems in the archive, and we’d love to hear about the fascinating journeys you find yourself on.
Jo Kent, Business Analyst, BBC Radio & Music Multiplatform.