How smartphone or tablet companion devices might enhance and interact with programmes on the TV
Project from - present
What we're doing
We want to make it possible for your smartphone or tablet (companion devices) to interact with your TV when watching broadcasts or using BBC iPlayer. Apps on your device will be able to synchronise to the TV show and enhance the viewing experience with additional sound or video, background information, suggestions of other programmes or opportunities to participate or play along.
Why it matters
Many people already use companion devices (such as smartphones or tablets) while they are watching TV. Sometimes they use those devices to enhance what they are watching - to discuss it on social networks, say, or to use a mobile or tablet application tailored to a specific programme.
For many TV programmes there is a wealth of background information, learning resources, play-along experiences and extra content. We make this available via the Internet and through the BBC’s website and apps. If we can make these things easy to find, then more people will be able to use them.
Broadcasting reaches a large audience and gives everyone the same thing, whereas a more personalised media experience often needs to be delivered over the internet. With synchronisation functionality in TVs and mobile apps, personalised media can be combined effectively with broadcast giving you the best of both worlds. Friends or family watching TV together could even each have their own different personal experience. For example: perhaps only one person in the room wishes to listen to Audio Description.
How it works
This has been a complex, wide-ranging project, in which several different teams from BBC R&D have worked together over a period of several years.
TVs are increasingly becoming “connected” devices that are able to run apps such as iPlayer. Their Internet connectivity opens up the potential to communicate with other devices in your home. Using this, an app on a smartphone can synchronise with the TV and take you straight to that extra content or play along experience with a single click or tap.
Early on, we experimented with technologies such as audio watermarking and ran pilots to experiment with play-along formats. We devised protocols, such as Universal Control API, to enable us to explore and understand what might be possible if TV and companion devices directly communicate and control each other. We also thought a lot about the way viewers were starting to use multiple screens and what might go into creating an Orchestrated Media experience.
We have also been working with the industry to put in place the technical standards that will be needed to make this work. We have developed specifications for synchronisation mechanisms with DVB and have contributed to the major new “companion screens” and “media synchronisation” features in HbbTV 2. A standards-based approach is vital for open platforms such as Freeview where the viewer can choose between many makes and models of TV.
Applying the Technology
With standards in place, we have also worked to ensure HbbTV technologies become a part of TV platforms such as Freeview Play. This will enable our audiences to benefit from these technologies.
We have developed and shown demonstrators proving that HbbTV can enable novel and improved user experiences, such as delivering audio description and other synchronised experiences on a phone or tablet or personalising broadcast.
Our work also includes planning for how BBC systems and metadata need to evolve. We are also expanding our understanding of user behaviour to ensure that the technology delivers the right capabilities. User experience studies and collaborative projects, such as 2-IMMERSE, help us do that and gain from the experience and expertise of others.
Some of our early work
Some of our early work included:
Universal Control API - a RESTful web API that enabled both companion screen experiences and new ways to meet accessibility needs. This was published as a White Paper.
UCMythTV – an open source implementation of Universal Control API for MythTV.
Commentator - a prototype mobile app that demonstrated Audio Description or a director’s commentary that was synchronised to the TV using Universal Control API.
An enhanced Autumnwatch Companion – the original prototype website displayed relevant facts and visualisations on tablets or laptops during an episode of Autumnwatch and we extended it to use Universal Control API to enable skipping between chapters. (2010)
The Secret Fortune closed pilot – where audio watermarking technology was trialled to allow 200 viewers to play along with the Saturday night quiz show to see if they could do better than the contestants in the studio (2011).
A personalized BBC homepage - Universal Control API was used to personalise the homepage with links and information about the TV programme being watched.
TV-controlled Daleks - toy Daleks were made to come to life, using Universal Control API, every time their larger cousins appear in an episode of Doctor Who. Ex-term-i-nate! And you thought it was the Daleks who controlled TV. (2011)
The Antiques Roadshow play-along experience – where we provided technology advice for this public trial of a play-along experience where you could try to guess the value of the antiques! (2013)
Standards we have contributed to:
A DVB specification for "companion screens and streams" ("DVB CSS") – a standard that enables frame-accurate synchronisation between companion and TV. It has been published as ETSI TS 103 286
The HbbTV 2 specification - this includes companion screen interaction and synchronisation using DVB CSS.
Resources and Open Source Tools
We have also released a variety of resources to help the industry with testing and development, including example streams and applications and open source libraries and tools, including:
This project is part of the Broadcast and Connected Systems section
People & Partners
Hybrid broadcast broadband TV (or “HbbTV”) is a global initiative aimed at harmonizing the broadcast and broadband delivery of entertainment services to consumers through connected TVs, set‐top boxes and multiscreen devices. The HbbTV specification uses elements of existing specifications from other standards including OIPF, CEA, DVB, MPEG-DASH and W3C.
an industry-led consortium of over 200 broadcasters, manufacturers, network operators, software developers, regulatory bodies and others in over 35 countries committed to designing open technical standards for the global delivery of digital television and data services