Research & Development

Posted by Max Leonard, Bill Thompson on , last updated

If you take a look at the back of a BBC staff ID badge, you will find a bold series of reminders of our core values. Chief amongst these is that audiences are at the heart of everything we do.

In Research & Development, we are using an innovative technology called a Personal Data Store to explore the types of services we could create if carefully protected audience data were at the heart of our underlying technology stack.

"We will need to be cutting edge in our use of technology to join up the BBC, improving search, recommendations and access. And we must use the data we hold to create a closer relationship with those we serve."

Tim Davie, BBC Director General, September 2020

Interactive Presentation - BBC R&D Showcase 2020

In this interactive experience discussing the use of personal data you can choose what you would like to know more about from our engineers.

New Super Powers

Building a closer relationship and tailoring the BBC's offering to better suit the individual needs of our audience doesn't have to mean filter bubbles and auto-generated news feeds. Improved accessibility with bigger subtitles for those with poor sight, local news and weather automatically located for you, or support for more advanced object-based services like perceptive podcasts and personalised augmented reality - all of these could be possible.

Doing this means using personal data, but the information needed may be sensitive - few of us would want to share our optical health records with a media organisation so they can get the right-sized subtitles.

We have been looking at alternatives to storing user data in large-scale databases - 'data lakes' - in ways that respect people's privacy but can still support rich data-driven services, while delivering real value to audiences.  The model we are exploring could offer many benefits, from reducing data storage costs to making severe data compromises less likely. It could also let audience members make full use of the provisions for data portability in current data protection law as they move between online services.

BBC R&D - Introducing the BBC Box

BBC R&D - Personal Data Stewardship

BBC R&D - Behind the Privacy Notice: Audience Data in Personalised Experiences

Putting audiences at the heart: a user-centric application architecture

The key is a technology called a Personal Data Store (PDS) which securely stores a user's data. They have been around for a small number of years now, and there are many different flavours. Some are like bank vaults where you store your data until you need to send it to a service to use, while others allow programmes to be run inside the PDS so that the data never leaves it.

The BBC Box and accompanying app on a tablet.

We began experimenting with this technology back in 2018 with the BBC Box project, and we have been working since then to address scalability issues and ways to make the user's 'data journey' understandable. We are currently using an open-source tool called Solid, originally developed by web creator Sir Tim Berners-Lee. Solid stores user data in 'pods' and we have used these to build a user-centred data storage system and a prototype media discovery tool to test our ideas in practice.

Only the user and the applications they permit can access the data in the pod. Any user activity generated through the use of services (posts, photos, play history, etc.) is saved back to the user's encrypted pod, rather than to the unified databases of the service providers.

In our first demo, a user creates a new data pod on the BBC system and then links their BBC and Spotify user accounts to pull in some of their media play histories. This data is then processed on the user's device to create a media profile, which is used to search against BBC News, music, podcasts and programme archive to allow users to find content related to their favourite artists.

It's a simple but effective demonstration of a secure cross-service application. At no time does the BBC get to see the user's Spotify data, and Spotify does not receive a copy of the users BBC data, as all data processing is done in the app. As well as keeping data safe, it means if you want to move to another service, your data is already under your control, and you don't need to ask for it.

We are also adding some open data sources to the mix, like weather and environmental datasets, and linking with third-party applications through a data exchange layer. In time we hope to create new conventions and standards to make interoperability between service providers simpler.

Where next?

Our work to date has shown the technical feasibility of this approach, but that does not guarantee adoption, especially as advertising models currently rely on extensive collections of user data in one place and under one organisation's control. Startups and big-tech sponsored ventures are now emerging, getting a handle on the problem of standardising data in a world of decoupled data stores. Other measures such as Apple's new anti-tracking features in iOS 14 are positive steps towards usable, meaningful privacy control.

Organisations such as ours that care about the privacy and agency of their users can start to design internal systems to prepare for a de-centralised future, instead of relying on surveillance technologies and centralised stores to be able to deliver our services.

We continue to work with others to share best practice and technologies and collaborate on genuinely new offerings that allow people to make use of the incredible utility of their data without having to relinquish the innermost details of their personal lives.

Tweet This - Share on Facebook - Share on LinkedIn - Share on Reddit - Share on Hacker News

BBC R&D - Introducing the BBC Box

BBC R&D - Personal Data Stewardship

BBC R&D - Behind the Privacy Notice: Audience Data in Personalised Experiences

SlashGear - BBC Box personal cloud aims to rewrite the rules on internet privacy

BBC News - Personal online data to inform BBC audience choice

BBC R&D - The Ethics of Personal Data

BBC R&D - Data Science Research Partnership

Broadcast - BBC tackles data safety with ‘disruptive’ device

Gizmodo - BBC Box personal cloud aims to rewrite the rules on internet privacy

BBC R&D - Privacy on TV

This post is part of the Future Experience Technologies section