I lead the engineering effort that delivers IP based audio and video services to BBC iPlayer and BBC iPlayer Radio. I want to follow up on the discussion on Jim Simmons post about Audio Factory and more specifically the impact that it is having on internet connected devices.
The BBC started streaming internet Radio back in the late 1990’s and since then has made a number of changes to its audio delivery systems to keep pace with the latest technology. Whilst some consolidation has taken place over the intervening years, until recently we were still maintaining a number of disparate systems with diverse operating models delivering to a number of different standards at great expense.
These systems were also old, required constant attention and failed much more often than we were happy with. Audio Factory was borne out of the need to do something much better to deliver our internet radio stations and also bring it in-line with the work that we have done in recent year’s improving our video systems ("Video Factory").
We wanted to make all of our delivery chains more resilient, improve audio quality across all stations and make access to those stations more consistent across the vast and diverse landscape of devices that we now need to support.
As Jim Simmons has detailed in his earlier posts, in order to do that effectively we have had to make some hard choices in what formats we could support. Some of those hard choices have inevitably affected some of our audience’s ability to listen to BBC radio.
The internet video and audio industry is increasingly standardising around chunked HTTP delivery – this includes formats such as Apple’s HLS, Adobe’s HDS, Microsoft’s SmoothStreaming and the emerging open Mpeg DASH standard.
Delivering in these formats has some great advantages for the BBC. Firstly, HTTP is the simplest form of internet delivery – it is how the vast majority of web sites are delivered. Aligning audio and video delivery with this is much cheaper as it doesn’t rely on proprietary streaming formats that require specialist delivery mechanisms. Instead we can make good use of much of the work that has been undertaken in optimising and caching the delivery of HTTP. It also makes it cheaper for us to deliver to the widest range of devices as we can encode the audio once and use a much cheaper ‘re-packaging’ method to deliver in the various streaming protocols.
Unfortunately this shift in the industry leaves some technologies in the shade. Windows Media Audio (or WMA) has been a mainstay of the BBC's streaming portfolio for many years, but is now unsustainable. The radio industry, certainly in the UK, has already largely abandoned it and soon the few Content Delivery Networks (CDNs) on whom we rely to deliver the content are bowing out from supporting it also.
This left us with a quandary – how do we effectively support the small community of internet connected radio users many of whom rely on the WMA format. We knew that use of Windows Media accounted for a small percentage of our overall internet listening (between 2-5%) but knew very little about the devices that were using them.
Very few of the devices announced themselves to us in an identifiable way, so despite a large amount of effort to identify the users we were unable to in the vast majority of cases. The situation with the few streams that we made available to internet radios as ShoutCasts was not much better. Our serving logs were cluttered with failed requests for the media, this meant that a large chunk of the traffic we were delivering from our ShoutCast servers was in fact a ‘forbidden’ response, due to requests from devices that didn’t understand how to communicate with our systems correctly. Clearly, spending money on delivering nothing is not a good use of licence payers’ money.
Over a year ago, we approached as many of the manufacturers, aggregators and other related groups as we were aware of to inform them about the changes. Some of these manufacturers have responded by making changes, some were unable to (due to limitations in their hardware) and others have decided to do nothing. We continue to work with these groups and are keen to engage directly with manufacturers or software providers who are inconvenienced by these changes.
I wanted particularly to try and address some of the main concerns raised in your comments on Jim Simmons’ post.
Radio 3 HD – unfortunately Radio 3 HD is no longer available as a ShoutCast stream, but is available in our chunked HTTP formats. This means that some internet radios will get a reduced quality experience as they will switch to our MP3 ShoutCast offering which is not available in HD sound. Many though will be able to switch across to the new formats either immediately or in the coming months as the manufacturers and aggregators roll-out updates.
Radio for the blind and partially sighted – It is particularly unfortunate that some devices targeted specifically at blind users have been affected by these changes. We did approach a number of the device manufacturers and software providers in this sector to advise them of the changes, but are conscious that this information did not reach all the necessary groups. We are now trying to contact these groups afresh to offer what assistance we can to get some of these devices back up and running.
Radio 5 listeners worried about sport being blanked from our streams – Our new ShoutCast streams are unrestricted which means that we can make them available both in the UK and globally, this significantly simplifies how they are served, but does mean that users whose devices can only receive the ShoutCast stream and not one of the new HTTP streaming formats will hear a version of the stream which will blank sport where the BBC does not hold international rights. The new HTTP streams are available in both UK and international variants, which allows those in the UK to continue to enjoy the Sport that we broadcast as usual. We are investigating to see whether there is anything we can do to return access to the UK streams for those who are affected by this
Issues with the new MP3 ShoutCast streams – It would appear that we have been having some teething trouble with our new MP3 ShoutCast streams, which has resulted in some buffering and inconsistent playback for some users. We have already put in place a number of measures to improve the situation here and will be monitoring closely for any further issues. We are also aware of some issues surrounding how some devices and aggregators handle failure scenarios. As failures can periodically occur, we have a pair of resilient streams that allows for one of the pair to have a problem without impacting the audio being delivered to the user. Unfortunately we are aware that not all devices and aggregators are able to handle this behaviour correctly – we are looking into ways of mitigating the impact of this.
Logitech Squeezebox – We’ve had lots of contact from disgruntled Logitech Squeezebox owners, I am pleased to say that I believe that most of the concerns here are about to be addressed thanks to the work of some of the active Squeezebox community and from Logitech who are now talking to us about supporting our Shoutcast streams. For Squeezebox users in the UK this should result in them being able to access not just Radio 3 in the highest quality, but also across all 57 of the radio stations currently supported by Audio Factory.
I'd like to apologise for the inconvenience that some of you have undoubtedly suffered due to these changes. We are currently working to address your concerns in the ways I have outlined above, and will be publishing more blog post to keep you informed and get your feedback.