Posted by Tom Parnell on , last updated
BBC R&D's Tom Parnell writes about setting up a 3D audio array at the Royal Albert Hall to capture the world's greatest classical music festival in binaural audio.
I’ve been on attachment from BBC Radio as R&D's Research Studio Manager since February 2016, when I was tasked with adapting the sound for the CGI-created VR fairytale The Turning Forest. The piece began life as an audio-only drama, designed to show-off the potential of 3D sound over loudspeakers, and it exploited the huge body of spatial audio research undertaken in recent years by the BBC and its partners. My task was to revise the soundtrack so it would work as a full VR experience, viewed using a head-mounted display, with headphones providing dynamic binaural sound - in other words, headphone sound that gives a 3D spatial impression and updates according to the viewer’s orientation. The complex re-mixing process helped us to develop a workflow for creating professional object-based audio for game engines and highlighted some challenges that need overcoming to improve future VR productions.
The audio research team is now working to optimise The Turning Forest for different platforms while continuing to support broadcast productions, including helping to produce several radio dramas in binaural. We are keen to explore how binaural sound could benefit other broadcast genres, including classical music, and I was keen to apply some new technology to an area of broadcasting that I know well. So we jumped at the chance to rig a 3D microphone array at the Royal Albert Hall to capture spatial sound from 2016’s Proms Season.
The microphone array is the Schoeps ORTF-3D and uses its eight hypercardioid microphones to pick-up sound from all directions - in front and behind, above and below. The array uses the directional characteristics of the mics to increase the sound separation between each channel and uses the spacing between the mics to introduce time differences between the sounds that each mic picks up. These characteristics work, together with some clever binaural processing, to produce a 2-channel “mix” (or “render") that provides the headphone listener with a surprisingly realistic reproduction of the 3D soundscape around the mic array. This is similar to recordings made with a dummy head microphone but, crucially, our object-based technique allows us to render these audio mixes for different playback formats, not just binaural.
Radio 3 has a large complement of microphones already rigged at the Royal Albert Hall so with valuable help from their operations team we rigged our additional 3D mic array in a suitable position (just behind and above their main pair of spaced mics) and set about capturing multi-track recordings of the first few Proms.
After the BBC Symphony Orchestra’s poignant rendition of the Marseillaise, the season began with Tchaikovsky's Fantasy Overture ‘Romeo and Juliet’ which provided a perfect orchestral texture to start mixing in binaural. I set about optimising the “sound” of the 3D mic array which is the main element of all the Proms binaural mixes. And in the same way that close mics are added to the broadcast stereo mixes I used a lot of the other Radio 3 mics to augment the sound of the 3D array. By applying appropriate delays and accurate 3D panning, these "spot" mics provide subtle focus to the direct sound elements of the mix and help to produce a more aesthetic musical balance. But with the Royal Albert Hall as our recording space I was keen to retain the spaciousness of the acoustic and, even with a good representation from the 3D mic array, I needed to add signals from the Hamasaki Square mic array (positioned high above the centre of the arena and used in the TV balances) to consolidate the reverberation of the hall in the mixes.
There is often debate among classical music balancers about how "realistic" versus how "musical" a mix should sound and this applies, if not to a greater extent, to a 3D mix too. With these binaural mixes I’ve tried to faithfully represent the musical performances (and physical layout of each ensemble) while also giving a flavour of how the Royal Albert Hall acoustic sounds with different ensembles. This works well in 3D as the sounds are positioned around the surface of a sphere rather than along a straight line between the left and right channels of a stereo mix.
With traditional dummy head mic binaural recordings the listener is deliberately placed in a particular position within the soundscape, but I’ve tried to avoid this approach; it might be interesting to hear parts of Schubert’s Great Symphony from within the double bass section but not the whole symphony! And while the experience of the Royal Albert Hall Prommers is often the best in the house, I was keen with these binaural mixes to provide a more generic, musical perspective on each piece, much like the Radio 3 broadcasts. Having said that, I’ll be experimenting in future with some of the close-miked Proms - fancy hearing a jazz orchestra with the band completely surrounding you?
I’ll be using the concerts towards the end of the season to trial new artificial reverberation and 3D room simulation tools in the binaural mixes. Applying reverb will help to further soften the addition of spot mics in the mixes and inform our future spatial audio productions, especially when creating 3D room sounds in radio dramas. We’ll also be using these 3D Proms mixes to test our automatic rendering of object-based mixes to different playback formats; we can use the same mix to render a different version, for example, for stereo listening, or a 5.1 surround system, a loudspeaker set-up with height channels, or even a dynamic binaural mix which renders as you “listen” around the soundscape (much like looking around a 360 video or a VR experience).
This post is part of the Immersive and Interactive Content section