BBC R&D

Posted by Nick Hanson on , last updated

In 2019 BBC Research & Development commissioned the personalised object-based documentary, Instagramification. We've previously written about the making of the pilot, but now, following extensive user testing, we can share some of the wider lessons specifically around the editorial considerations and the audience's response. The intention is to help us better understand the challenges and opportunities around the use of personal data to customise factual content to individuals.

To recap, the documentary was made by Spirit Media using our StoryFormer tool. Different versions of the same story are served up based on the viewer's personal preferences. These include which UK nation they live in, whether they like celebrity culture, sport or technology, and whether they would prefer to be 'informed' or 'entertained'.

Our research questions were:

  • Through StoryFormer, can we create object-based experiences that allow for greater personalisation of the same story?
  • Can we create coherent content experiences that fit together in a meaningful way?
  • Does it lead to efficiencies such as 'sweating the assets' or are we just creating more content for fewer eyes (as with interactive video)?

Some of these questions were partly answered during the development and production process, and we addressed them in our earlier blog post. However, to answer these in full required further testing with audiences.

BBC R&D - Behind the Privacy Notice: Audience Data in Personalised Experiences

BBC R&D - Object-Based Media

We did this in three stages. Firstly we looked at the responses to the survey questions on BBC Taster. The questions we asked for our research demonstrated a positive response to the format:

A table of survey results, showing the top answers were 40% of people Strongly Agreed the tone suited their mood, 33% that the content suited their interests, and 38% felt it flowed naturally.

Moreover, 67% of respondents to the survey believed the BBC should be making "more stuff like this". It has also been the top-rated pilot on Taster since launch with an average rating of 3.6 out of five, with 36% of users giving it a five-star rating.

However, Taster can admittedly be a self-selecting user group who are actively open to new formats. Also, although these results appear to show that the audience responded positively to the pilot, they don't tell us precisely what they liked or if they have fully grasped the true nature of the personalisation.

So we conducted a second study, this time using the online survey tool UserZoom. We asked 60 participants with a 50/50 male/female split, all between 18-34, from mixed socio-economic backgrounds and all familiar with Instagram, to record themselves watching the documentary and comment out loud throughout. We followed this up with a series of survey questions:

Survey results: 52% agreeing they enjoyed the documentary, 38% agreeeing they understood how it adapted to their preferences, 32% being unsure if it was better for being personalised, 30% unsure if they would rather watch a non-personalised documentary, 33% being unsure if they'd be comfortable with the BBC using their data, and 30% agreeing they would like to see more of this on the BBC.

Generally speaking, the UserZoom results were also positive. However, in terms of allowing us to understand the value and effectiveness of the personalisation, the survey was inconclusive. For example, we asked users if they preferred a personalised documentary to a 'traditional' non-personalised one, but of course, our users had nothing to compare like-for-like (hence no clear opinion). We will revisit this point in the third and final study.

The comments made by users were, for the most part, extremely positive:

"The choices I made were absolutely reflected in the content, and that's what kept me watching."

"100% reflected what I would want or expect to see."

"This has taken interaction to a whole new level."

However, Instagramification did not win everyone over. Some had valid criticisms:

"I think the questions need to go into more depth so it can be properly tailored."

"I didn't feel it was personalised. I said I wasn't interested in celebrity culture, and there was quite a lot of it in the documentary."

"Focusing on select topics within the documentary is very dangerous…people will not learn outside their world."

The final comment alludes to the potential for personalised content to create echo chambers or reinforce stereotypes, an incredibly important and valid point raised by several users. Our study is not exploring this in-depth this time, and we will certainly revisit this, but for now, we're focussing on how we tailor content based on our research questions stated above. The comments on depth and level of personalisation, however, are directly relevant to our study.

When watching people explain what they liked or didn't like about Instagramification, it became clear they didn't always realise how or when the personalisation was happening for them. For example, if the user stated an interest in celebrity culture, they are often happy to see many celebrities featured in the documentary. If they said they were not interested in celebrity culture (as per the comment above), they were often disappointed and felt it wasn't tailored to them.

A composite graphic representing Instagramification.

In fact, all versions of the documentary feature celebrities to some degree, the difference is in the length and depth of those stories and the tone in which they are covered. In short, audiences often made the wrong assumptions about what was tailored to their preferences, even when they felt very positive about what they were seeing.

On the whole, we were satisfied that we had created a documentary with multiple coherent pathways, all of which made narrative sense. We were also happy that we had done this without making hours and hours of content that would not be seen by most viewers (as is the case with interactive video). But we still wanted to better understand how we use the data from our audiences to create experiences that are meaningful to them as individuals.

Our third and final study involved working with design agency Sigma. We recruited 10 participants for one-on-one 60-minute qualitative research sessions. Each participant watched the video, selecting their personal preferences and followed up with a conversation about what they had just seen. They then watched again but chose the exact opposite of their original preferences. This was followed up by another discussion. This method allowed us to make comparisons between what the user thought about their personalised version and an alternative version.

The results again saw a generally positive response to the documentary with an appetite for personalised content, though significantly, most emphasised they would want transparency and control over how to do this. The tone and language of the two presenters had a big impact, and the structure and flow of the story were generally positive. Significantly, most (though not all) participants felt the first version they viewed (the one that used their data) was preferable to the second version they saw. This demonstrates that through object-based media, we can create different versions of the same story where each individual receives a version more relevant to them.

However, we're still not 100% there. As with the previous study, initially, participants didn't work out exactly how or which parts of the documentary had been tailored to them. Significantly, even when it was tailored to them, it wasn't always done in a way they felt had a positive impact. For example, those who stated they liked sport received an extended version of a story on footballer Cristiano Ronaldo. However, when questioned, some said that just because they liked 'sport' doesn't mean they like 'football'. Or just because they like 'football' doesn't mean they like Ronaldo. Or just because they like Ronaldo as a footballer doesn't mean they want to hear more about his family life as was the case in Instagramification. This demonstrates perhaps the most significant editorial challenge for object-based storytelling – how can we translate the data into meaningful content?

BBC R&D - Making a Personalised, Data-Driven Documentary

BBC R&D - StoryFormer: Building the Next Generation of Storytelling

BBC R&D - StoryKit: An Object-Based Media Toolkit

Summary

What does this all mean? It's clear we need to get better at knowing what data to collect. We also need to get better at understanding how that data translates into meaningful content experiences.

Looking back at our research questions, success in creating scalable and repeatable personalised factual content is finding the 'sweet spot' between some potentially conflicting things: retaining the story structure, providing meaningful personalisation and getting value from the assets.

As we stated previously, personalised storytelling doesn't need to be about creating radically different versions for each viewer. Instagramification branches off into different routes through the story at the point of making one big decision, but then works through a series of 'micro-personalisations', where lots of small changes combine across the course of the story for a more substantial effect.

The pilot was largely successful in demonstrating how object-based media can create a different version of the same story around the needs of diverse audiences, but it also surfaced the challenges of doing so. There is still a long way to go, but Instagramification represents a big step in the right direction.

Tweet This - Share on Facebook

BBC Taster - Try Click's 1000th Interactive Episode

BBC News - Click 1,000: How the pick-your-own-path episode was made

BBC R&D - Making a Personalised, Data-Driven Documentary

BBC R&D - Object-Based Media

BBC R&D - StoryFormer: Building the Next Generation of Storytelling

Tech Crunch - Interactive content is coming to Walmart’s Vudu & the BBC

Broadcast - BBC creates interactive fact-ent show

TVB Europe - BBC announces first full-length interactive programme

BBC R&D - StoryKit: An Object-Based Media Toolkit

BBC R&D - Delivering Object-Based Media at Scale with Render Engine Broadcasting

BBC R&D - How we Made the Make-Along

BBC R&D - Cook Along Kitchen Experience

BBC R&D - Story Explorer

BBC R&D - Newsbeat Explains

BBC R&D - Creating Adaptive Stories with Paper Tools

The Next Web - How the BBC is building the future of broadcasting by blowing it up into little bits

This post is part of the Future Experience Technologies section

Topics