BBC R&D

Posted by Graham Thomas on , last updated

The Natural History Museum in London have just opened an interactive film called Who Do you think you really are? in the new David Attenborough Studio lecture theatre. The film was produced in conjunction with the BBC Natural History unit, and they were keen to make it more than just a conventional video presentation. They approached us at BBC R&D to see if technology we'd previously developed for adding 'virtual' graphics to TV could be adapted to produce an augmented reality element for the film. The end result is an innovative museum experience that makes use of our camera tracking technology, in conjunction with specially-designed handheld displays and rendering software developed by other partners in the project, to bring extinct creatures to life using augmented reality.

The Interactive Film exhibit at the Natural History Museum Copyright Natural History Museum- Rights Reserved The Interactive Film exhibit at the Natural History Museum

Copyright Natural History Museum- Rights Reserved

The film, which looks at evolution, uses the handheld display with a webcam to give each member of the audience their own window on the past. Animated 3D models of extinct creatures are combined with live video from the webcam in the handheld to create the appearance that the creatures are wandering around the studio in front of the audience. The camera tracking technology allows the system to render a view of the creature that matches the handheld's camera view by accurately calculating the camera's position and orientation within the studio. This gives the audience an opportunity to use their handheld display to track the creature as it moves around them.

The Interactive Film show at the Natural History Museum uses special tablet devices to allow visitors to experience a unique viewpoint on virtual exhibits

The Interactive Film show at the Natural History Museum uses special tablet devices to allow visitors to experience a unique viewpoint on virtual exhibits Copyright Natural History Museum- Rights Reserved

Augmented reality applications on hand-held devices are becoming increasingly common, but most rely on technology like accelerometers, solid-state gyros and compasses which do not provide the solid registration between real and virtual content that was needed to make this experience really believable. We have previously developed approaches that rely on image-based tracking, for example the free-d system (used in 'virtual studio' applications like graphics for Election night; it uses circular barcoded-markers on the ceiling of a TV studio, viewed by an upward-looking camera mounted on the studio camera), or the sports graphics system used on programmes like Match of the Day, that tracks lines on a football pitch. These provide estimates of the camera position and orientation that are accurate enough for very good registration, but they need special markers or lines to be present. Tracking from arbitrary image features is possible, but can be difficult, particularly in a situation where the scene is poorly-lit or the lighting is changing. To make an image-based approach work in the context of the museum, we needed to find a way of putting distinct visual markings into the scene that the tracking system could see but that would not detract from its appearance to the audience. We came up with a solution that used infra-red LEDs, which are visible to the webcams but not to people. The LEDs are concealed within the structure of the David Attenborough studio, in accurately-known positions. We also devised a method for the system to work out which LED was which.

More information about the film can be found on the Natural History Museum's website, and some of our other work on virtual graphics is described on the R&D website.