Research & Development

Posted by Emma Young on , last updated

BBC R&D have launched a user survey to find out how people feel about the concept of machines being able to express emotions. We hope the survey will help us understand how people in the UK are 
using voice technologies in their daily lives and the kind of emotions that people experience while interacting with current voice assistants (e.g. Alexa, Google Assistant or Siri). We’re looking to gather responses from a broad range of voice device users and non-users.

(Update: The survey has now closed and we have removed the links to it from this post.)

Why Now?

In early 2018 it was reported that 7 million UK households were using smart speakers, with that figure predicted to rise to 12.6 million by 2019. With devices like Amazon Echo and Google Home featuring high on the gift list for many of us last Christmas, that figure may well be considerably higher. So, many of us are now living with voice-driven devices, both in our homes and built into our mobile phones, but why is this important to the BBC?

A lot of BBC content is well suited for consumption over smart speakers (BBC Radio, BBC News, BBC Sport, BBC Weather, BBC Sounds) and BBC teams are working to improve the delivery, discovery and navigation of this content for voice-driven devices. As these devices become more widespread, more of us are using them to consume content and as a public service broadcaster, it’s our duty to ensure that we’re providing great interactive experiences and new forms of content that are optimised for the wide range of devices our audiences use.

Here at BBC R&D, we are always looking to the future and since 2016 our Talking with Machines project has been exploring new forms of voice-interactive content. This work led to the development of Orator, a set of tools for writing and playing interactive stories on voice devices, now used and extended by the BBC Voice team for products such as the CBeebies Alexa skill. Orator was originally created for R&D’s work on The Inspection Chamber - ask Alexa to enable it! Further work led to our recent release of interactive drama The Unfortunates, which you can try for yourself on BBC Taster or by asking Alexa.

As the Talking with Machines work continues (exciting stuff happening – watch this space!), a few of us started thinking about the sorts of experiences that could be possible if voice devices were able to express their own emotional states. We’ve seen moves in this direction with Amazon’s Speech Synthesis Markup Language (SSML), which gives Alexa’s voice some expressive elements such as emphasis and intonation, however 95% of communication is non-verbal and it’s body language, facial expressions and non-verbal vocalisations (e.g. hesitation, laughter and intakes of breath) that speak volumes in human communication. This leads us to wonder about non-verbal expression for emotional machines.

BBC R&D - Top Tips for Designing Voice Applications

BBC R&D - User Testing The Inspection Chamber

BBC R&D - Prototyping for Voice: Design Considerations

We know what you’re thinking - machines don’t have emotions, right?

Well what if they did…

Imagine a smart speaker that gets embarrassed when it can’t find things; tired after a busy day; saddened by bad news; excited about visitors; or is feeling cold?

Perhaps emotionally expressive devices could provide more useful cues for users - if so, this opens up a wide range of possible applications. It could provide a supportive form of interaction to assist with isolation in the elderly and help with awareness around looking after ourselves and living well (“Alexa’s getting tired… oh my - look at the time! I best be off to bed”). It could assist with the frustration that we sometimes feel when using technology, for example if we can see that a machine is working hard to complete a task for us or if it fails to understand a command, it might encourage some patience on our part and stop us from wanting to throw our device out the window!

It could also be a useful indicator that a child has spent too much time with their tech - Apple addressed this with their Screen Time tool but Janet Read, professor of child computer interaction at UCLan suggests that more useful cues are possible if computers were to behave more like humans:

“Maybe the computer could have a hissy fit, or it could slow down, or stop interacting or be naughty. That kind of interaction could be more helpful to a child’s development because it reflects our own instincts and behaviours. If the computer decides that 20 minutes is enough, or that we seem too tired to play, it could just shut down – and, in doing so, help us to learn what the right time to switch off feels like.”

An Amazon Echo placed on a wooden table.

As voice devices get better at natural language processing and sentiment analysis, we’ll see new applications for smart speakers emerge that move beyond simple command and control (e.g. using our voice as a remote control for radio) towards an intelligent system that we’ll be able to talk with in a way that is more natural, like human conversation, and more responsive and adaptive to user interaction.

A recent study found that 51% of young people in the UK believe that machines are (or will soon be) emotionally intelligent enough to form relationships with humans. At BBC R&D we believe that along with a shift to more human-like interactions, voice devices and other systems will get to a point where they can accurately sense mood and emotion and respond accordingly. Imagine a voice assistant that was able to read how you were feeling and change its behaviour, tone of voice or functionality to better suit your mood, in the way humans do – what if Alexa could give fast and to-the-point answers when you’re in a rush; interact cheerfully and playfully when you're feeling upbeat; and be soothing, restrained and low-key when you’re feeling tired or blue.

We think this is a really interesting concept and are excited about exploring this area with users. Your responses and insights will help inform and shape our work on Emotional Machines. The survey results will feed into a series of workshops we’ve got coming up where we’ll be exploring how machines might express different emotions through the modalities of gesture; sound; light and colour.

Update: You can now read the results of the survey in our report

Following on from the workshops we’ll be building some prototypes and will be taking them into the wild to run some user tests so watch this space for updates!

Tweet This - Share on Facebook

BBC R&D – Talking With Machines

BBC R&D - The Unfortunates: Interacting with an Audio Story for Smart Speakers

BBC R&D - The Inspection Chamber

BBC R&D - The Mermaid's Tears

BBC R&D - Responsive Radio

BBC R&D - Audio Research

Amazon Alexa Developers

Google Developers - Assistant

Apple Developer - Siri + Apps

BBC Academy - Talking to the internet: Digital assistants and the media

This post is part of the Internet Research and Future Services section