Using machine learning to create new types of metadata and open up the BBC archives
Project from - present
What we're doing
With this research we are developing systems that can recognise, and more importantly understand, what's in a programme (eg people, places, objects such as cars or Daleks), what is happening (are characters talking or shouting at each other? What are they saying? Is someone running?) and what is the mood, or feeling of the programme.
Why it matters
The metadata we currently have helps you find a programme if you know either its title, when it was first broadcast or members of the cast.
But what if you didn't want to search by these terms? What if you wanted to browse the archive, or find, say, an exciting spy drama similar to Spooks, or a satirical comedy show about politics?
In order to solve these questions we need new metadata.
How it works
We are using machine learning techniques such as image recognition, extraction of semantic meaning from speech and text, and analysis of music and audio to analyse programmes automatically.
Once we have generated all of this metadata about a programme, the next area of our research is to develop systems that store and index this a useful way, and develop ways to allow people to search for what they want most effectively.
We have prototyped a graphical user interface for navigating programmes by mood (see the blog post below for more details), which we want to test internally to see how useful a tool it is.
This project has also led to the MusicalMoods experiment in collaboration with the University of Salford and the British Science Association. Here's a video about that work:
We want to develop an overall "interestingness" index that combines the video and audio features automatically extracted from a programme. This could be used to identify significant moments in a programme, such as a goal in a football match, a particularly funny joke, or a dance routine).
As well as contributing to wider archive research, the vision, text and audio processing techniques we are developing have benefits for other areas of the BBC, where large amounts of data need to be automatically analysed and understood.
People & Partners