« Previous | Main | Next »

An Affective Interface for Mood-Based Navigation

Post categories:

Rosie Campbell Rosie Campbell | 12:00 UK time, Monday, 16 January 2012

Sometimes we need 'cheering up', 'calming down' or 'a bit of excitement'. Whether we notice it or not, we use affective (i.e. mood-based) language pervasively in life. Yet when we browse for media, we find content categorised into rigid, traditional genres, with searching restricted to factual metadata such as title or director. There is little allowance for relative scales or subjectivity.

The Multimedia Classification project in Archives Research is attempting to address this by answering the question of how we can use mood as a meaningful navigation tool. It certainly isn’t a feckless pursuit: if the BBC archive is ever to be made available to the public, we’re going to need some help finding what we want. From hundreds of thousands of hours of programmes spanning over 75 years, simply searching for ‘comedy’ isn’t going to get you very far! In fact, in our recent study, the majority of people said they would find it useful to be able to search by mood. I understand if you’re sceptical, we’re so used to conventional searching, it’s hard to imagine a useful alternative. However we’re not suggesting this approach will replace conventional methods, more that it will augment and improve them. It may even turn out that searching by mood is something people didn’t realise they wanted it, but once it’s there they’ll wonder how they ever managed without it!

So how does it work? The classification system automatically analyses programmes for a range of different video and audio features, such as luminosity, laughter and motion (see here for further details). The results of this are then used to assign each programme a rating on a set of mood scales. For example, a programme with a high level of motion but not much laughter might score 5/5 on the ‘slow-moving to fast-paced scale’, but 1/5 on the ‘serious to humorous’ scale, meaning it is quick but not very funny (a thrilling drama for instance). The advantage here is that it is then possible to compare different programmes based on their mood scores, allowing you to, for example, search for something ‘more exciting than Spooks’. Moreover, the combination of scores on each scale gives the programme a kind of mood fingerprint, so the system could recommend programmes with similar mood fingerprints to ones it knows you like.

We're currently in the process of researching how users might interact with this mood data. A prototype interface we have developed is a 2-dimensional scatter chart, with each axis representing one mood scale.

Mood Chart: helping users browse by mood

Mood Chart: helping users browse by mood

This allows users to view the relative moods of programmes as dots plotted on the chart. They can then either pick the mood they want using the sliders, or search using conventional means, using mood as a reference. It is hoped that this frontend will be integrated with BBC Redux (a BBC internal research tool) within a few months, as a preliminary trial to see how useful people find this type of affective interface.

Another area of interest is how mood varies throughout a programme. We’ve built a prototype system using the Universal Control API which allows users to view a dynamic graph of mood data on a second screen (such as a tablet PC or smart phone) that moves along in sync with a programme on a set top box.

Intelligent Skipping - Peaks and troughs in the graph show how the mood is changing

Intelligent Skipping - Peaks and troughs in the graph show how the mood is changing

This graph can display individual features (such as luminosity shown above) but for public use will most likely show an amalgamation of the many different audio and video features of the programme as analysed by the classification system, to provide an overall value for what we are provisionally calling ‘interestingness’. Imagine a kind of ‘intelligent skipping’ service, so that by tapping on peaks and troughs shown on the graph you can skip ahead to goals and car chases, or straight to the desired performance in entertainment shows.

There are a number of directions this project could take in the future. A more public-friendly user interface than a graph is desirable, and here there is scope for innovative interface design. The problem of mood subjectivity will need to be addressed; since something I find humorous may not be so funny to you (this became apparent in our user trials where many people rated a serious documentary on 70s fashion as very humorous…) The current view is that the system would take advantage of machine learning technologies so that it could adapt to your tastes: the user would rate the success of mood categorisations and the system would gradually learn, improve and become personalised.

Ultimately, the Multimedia Classification team hope to augment browsing, searching, skipping and programme recommendations utilizing mood-based metadata, and, with user experience in mind, bring you an effective affective interface.

Comments

  • Comment number 1.

    A good idea in principal but are you coming at it from the wrong direction? Are people in the least bit aware of their true mood at any point? Should the main research be about detecting the mood of the user, although that may be a bit Big Brother!?

  • Comment number 2.

    Our main priority at the moment is to detect moods of programmes in order to allow people to actively choose what sort of mood of programme they want to watch, not to suggest programmes based on the user’s current mood. Although detecting the mood of the user could be an interesting area of future research, simply matching a person’s mood to a programme might not always give the desired results – if you’re feeling sad you are likely to want to watch something happy or humorous to cheer you up, for example.

  • Comment number 3.

    Y would they care what mood we are if it wasnt for marketing purpose and selling. Internet is not a general place anymore every search result is tqrgeted and tailored to the individual doing the search. I would like to get more privacy. This mood thig is gonna make relevant results show according to personal mood, but I also want important info that is not relevant to me, I also want information that may be opposite to my view, information that may be different if I was in a different location. SITec

  • Comment number 4.

    Interesting. A couple of questions:
    1. Are not existing content genres good enough proxies for mood? Comedy, drama, thriller etc?
    2. From the work you've done is affective language a good indicator of mood when looked at in the singular or do you need to group (e.g. minimum threshold of mentioned terms for example) before it does become a good indicator of mood?
    3. Basic sentiment analysis often gets it wrong so I'm imagining there are lots of false positives (irony! black comedy!). Are there and if so, how are you looking to manage this?
    Cheers,
    James.

  • Comment number 5.

    A couple of questions turned into 3, sorry :/

  • Comment number 6.

    Interesting idea - I can see a lot of potential for something like this. How many metrics are you using for classification and do you have some system in mind for combining them all into a final decision. I.e. this program is "funny-intellectual" (funllectual?), or is it more of a Program A is funny where Program B is intellectual?

 

More from this blog...

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.