HTML 5 and timed media

Post categories:

Sam Dutton | 12:27 UK time, Wednesday, 12 August 2009

Comments (3)

HTML 5 work

HTML 5 is the next version of HTML, the markup language used on the web. Not all the details of the HTML 5 standard have been agreed, but many of the proposed changes and new features have already been implemented in existing browsers.

As part of our work on the P2P-Next project, we built a simple HTML5 demo that works in current versions of Firefox, Safari and Chrome: a sample of RAD's R&D TV with subtitles and chapter navigation. This will not work in current versions of Internet Explorer, nor earlier versions of Firefox etc.


What we built

This prototype plays video and audio without plugins, and allows jumping to chapters and 'scrubbing' within the content. It uses simple JavaScript framework to enable web page elements to be changed via individual HTML or CSS 'events', and for loosely-coupled publish/subscribe control of page components such as carousels. In particular, our JavaScript enables synchronised changes to HTML and CSS relative to a 'time parent', such as an audio or video clip, or even clock time. In addition, our solution needs to work with live events, whereby pages would be propagated in real-time.


Flash and other Rich Internet Applications (RIAs) provide something like this already via timeline scripting, but RIAs are 'black boxes', using compilers and obfuscators to hide code and data: great if you want to protect intellectual property, whereas we needed to provide a mechanism whereby data and the code acting on it were open and accessible. HTML 5 and the jQuery JavaScript framework gave us the tools we needed without requiring extra plugins or proprietary software.

Below we give more detail of the coding work done so far.

Synchronised media

On a web page it is sometimes desirable to make something happen relative to the current time of a video or audio track.

For example, subtitles change as a video plays.

YouTube Annotations adds something more dynamic: speech bubbles, notes and linked areas that can be set to appear in specific locations on a video, at specific times. This enables quite complex interactivity and navigation - in YouTube's example, a video about World War I leads to other videos about individual subjects such as tanks and aircraft, with extra notes that pop up as appropriate.

There are several open technologies available, or in development, for adding timed text (subtitles), but not more complex page changes and interaction. Annodex and parts of the Axmedis project go further in developing ways to integrate timed markup for annotation and indexing with streaming media. Annodex uses the CMML markup language, which can be either incorporated with, or referenced from, a stream.

The SMIL presentation markup language provides rich features for timed interactive media, and has been in use for more than 10 years, but it still has some limitations:

  • SMIL can be played in several media players, but browser support is patchy and the future of SMIL browser support is unclear
  • SMIL uses a presentation paradigm, and each presentation needs to be complete in itself, whereas our 'engine' needs to be event-driven, able to cope with individual live page changes, with viewers either joining a presentation part way through 'broadcast' or viewing the entire presentation after it has been published
  • existing SMIL implementations are oriented to elapsed (clock) time and user input events, rather than synchronising events with a 'time parent' such as the current time of a video2.

Having considered the alternatives, we decided that our 'engine' would be most flexible and accessible if we used JavaScript alongside functionality developed as part of the HTML 5 specification.

The JavaScript we wrote does a simple job:

  • build a list of page-change event objects, parsed from incoming JSON
  • activate or deactivate each event, given its start and end time, relative to the current time of the event's time parent, which in our demonstrations is a video).

The HTML 5 audio and video elements remove the need for player plugins, work like any other HTML element in terms of styling and positioning, and standardise the programming interface for playback control. Less well known is that these elements emit a timeupdate event (at a frequency adjusted to fit available processing and memory) which removes the need to poll a player for the current time position. This makes media scripting far more efficient, since there is no need to run a loop or use setTimeout. In tests run on several machines we found that timeupdate events are emitted regularly and frequently (particularly in Firefox), whereas polling a media player for current video time is unreliable.

The HTML 5 addCueRange method will provide native support for callbacks (or events) at the start and end of a 'cue range'. However the specification is still under discussion and does not appear close to implementation.

HTML 5 video in the field

HTML 5 media elements are now supported by current versions of Firefox, Safari and Chrome.

However, the implementation of the video element has been dominated by the need to standardise codec support, in the same way that browsers currently support the JPEG, PNG and GIF image formats.

Google demonstrated their commitment to both the MP4 (H.264/AAC) and Ogg (Theora/Vorbis) with the HTML 5 media element at the 2009 Google I/O conference. Firefox 3.5 and Safari both support the video element, though for different codecs. Dailymotion has created a demo site with several hundred thousand videos encoded using Ogg Theora.

The biggest and least predictable change may come from technologies such as Comet or HTML 5 Web Sockets. These enable data to be 'pushed' to browsers from servers rather than vice versa. Push makes sense, in that enables updates without polling, but it challenges the HTTP request/response model used on the web and raises a number of security and editorial questions. In terms of our application, events could be pushed to the client browser as they became available. For example, a web page showing a broadcast from a live event could include live updated information about the festival or the band and song currently playing.

We've done this work in conjunction with the P2P-Next project. We're using HTML5 in the project to sync video and display extra information about the content - it's early days for us on this and there are a number of serious challenges before this becomes anything near mainstream - if ever. We hope it's a useful demo and look forward to feedback.

Quick install/build guide for RadioDNS

Post categories:

RAD admin | 12:17 UK time, Sunday, 19 July 2009

Comments

Here's a quick list of how to get the RadioDNS application we released recently to build and run (on Ubuntu 9.04. Success reports for other distros or OSs welcomed.)

sudo apt-get install python-socksipy python-wxgtk2.8  python-dnspython python-setuptools python-dev

sudo easy_install objectlistview

wget -v http://github.com/BBCRadLabs/RadioVisDemo/zipball/master

unzip BBCRad*

cd BBCRad*

python radiovis_demo.py



And that should be it. Thanks for the useful posts on the radiodns.org list which summarised these steps.


RadioDNS Demo Application Released

Post categories:

Chris Needham | 13:30 UK time, Friday, 10 July 2009

Comments (1)

alpha_tag.gif

Following my previous post, Experiments with RadioDNS, I'm pleased to announce that the RadioVIS demo application is now available as open source software, under the Apache 2.0 license.

The application demonstrates radio station DNS lookup, querying for available services using DNS SRV records, and the RadioVIS protocol for displaying images and text messages. The code is written in Python and uses the wxWidgets user interface library.

Please note that this software is intended for those who want to find out more about the technical details of RadioDNS and RadioVIS, and not for end users. Also note that at the present time there is no RadioVIS service available for BBC radio stations. We are making this code available as part of our explorations with RadioDNS, and look forward to technical feedback.

To download, please visit our RadioDNS project page here. We'll put any updates to the code on the project page and announce them on this blog. The source code is also available on GitHub here.

For more information about RadioDNS, or to join the RadioDNS developer discussion group, please visit http://radiodns.org.

BBC navigation

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.