Testing mobiles: Mobile Compatibility Program

Wednesday 5 March 2014, 08:35

Paul Rutter Paul Rutter Test Manager

Tagged with:

Hello, I'm Paul Rutter, Test Manager for POD Test in Mobile Platforms, Programmes and On-Demand, BBC Future Media.

In my last blog post, "Testing BBC iPlayer for Android Downloads" I mentioned how our Android products are installable on over 3000 different devices. In addition to this vast range of devices, there’s also different firmware versions and operator customisations to consider. For iOS we too have hardware and firmware fragmentation to consider. That’s for native applications. For responsive web products the hardware/firmware considerations are even greater.

More and more people are using mobiles and the BBC has many products that audiences enjoy on an ever-increasing range of mobile handsets and tablets. We need to better understand, internally, what are our supported devices across our range of products and also to better inform our audience on what devices they can expect to get the best experience with on our products.

In addition, we’ve always worked with manufacturers to evaluate new devices that are yet to hit the market to make sure that they are technically capable of supporting our products and that the experience of our products on the device will be acceptable. We want to make this a more robust and formalised process.

To address these issues, we’ve rolled out a Device Compatibility Testing program in my team. We are calling it the BBC Mobile Compatibility Program.


What are we trying to achieve?

The program is expected to provide the following:

• A better understanding of what our supported Mobile devices are across BBC mobile native applications and responsive web products

• Evaluation of pre-production devices for compatibility in meeting technical requirements for BBC mobile products

• Evaluation of pre-production operating system updates

• Advice on classification of devices for product specific features

• Maintained results, fedback to manufacturers and the audience

• Initial focus is on Android, in particular BBC iPlayer, BBC Media Player, BBC iPlayer Radio and BBC Sport

We hope the program will cover all BBC mobile products in the future.

How devices are chosen:

In order to ensure the cost effectiveness of committing to support a device, we have set fair criteria each must meet to determine whether they should enter the BBC Mobile Compatibility Program.

Further information on how devices are selected for BBC iPlayer features (including the downloads service) is available on the BBC iPlayer help pages

Once we have run the MCP on all devices that meet the selection criteria, we will continue to work our way through other devices in order of popularity in the UK and BBC usage.

How is the Testing performed?

We are trying to run as much of the testing outside of our products. We have developed an Audio-Video (AV) Test Harness application that allows us to select and play representative video content, at different bitrates, on the device to perform our video playback evaluation.

We are working to integrate our DRM solution and add samples of our audio streams into the AV Test Harness. This will allow us to evaluate DRM video downloads support and audio playback evaluation outside of the respective products (BBC iPlayer and BBC iPlayer Radio).

Longer term we are hoping to offer access of the AV Test Harness to manufacturers so they can perform some level of compatibility testing themselves before submitting a device to the BBC.

In addition to the AV Test Harness evaluation, we also perform smoke testing of core functionality of the BBC iPlayer, BBC iPlayer Radio, BBC Media Player and BBC Sport products.

As mentioned above, longer term we hope to extend this to the full range of BBC native applications as well as responsive web products.

The output

Once testing is complete we update our results matrix and create a short report per device. This can then provide any or all of the following:

• Advice to Product Owners around device compatibility

• Recommendations to enable/disable particular product features

• Product defects to be reviewed by a product owner and prioritised within a development team

• Defects relating to the device and/or operating system itself which are fed back to the manufacturer

I hope this insight has been interesting and if you have any questions please feel free to comment below.

Paul Rutter is Test Manager, POD Test in Mobile Platforms, Programmes and On-Demand, BBC Future Media

Tagged with:


Jump to comments pagination
  • rate this

    Comment number 1.

    I am new to blogging so hope this gets through to someone in the know. I have a samsung galaxy note 8.0 which is an approved device. Have the BBC I Player app installed. When I go to a tv programme on BBC I player cannot see an icon for downloading. I can on my pc but not on my tablet. Can you please help and hope that you can send a reply to my e mail .[Personal details removed by Moderator]

  • Comment number 2.

    This comment was removed because the moderators found it broke the house rules. Explain.

  • rate this

    Comment number 3.

    Any idea how long it will take to add more devices to the iPlayer download list as currently it seems to be limited to Samsung, Sony and Kindle devices. What about the brand new Nexus 5, 10 or any of the other manufacturers out there that have ICS or above phones?

    At the initial rate of suggested roll out 1-2 per week there should be more than 50 devices on that list?

    After the initial burst of publicity (70% of Android app users able to use it from day 1 was seriously misleading but that was what many sites picked up on) it's gone very, very quiet.

  • rate this

    Comment number 4.

    Whilst some portions of this are welcome, you still appear to be taking a we know best attitude rather than letting users of new or less popular devices find out for themselves what works. I appreciate you have to consider the non-technical parts of the audience, but I think you could do a lot more to work with the more advanced users instead of frustrating them. Google now offers the ability to run an opt-in beta test through the Play store which could be one way of satisfying users and getting early feedback on new devices without the risk of a non-technical user accidentally obtaining this version.

  • rate this

    Comment number 5.

    quite right too.


Comments 5 of 7


This entry is now closed for comments

Share this page

More Posts

BBC News website: responsive design in beta

Tuesday 4 March 2014, 10:41

New search results page for mobile in beta

Thursday 6 March 2014, 09:53

About this Blog

Staff from the BBC's online and technology teams talk about BBC Online, BBC iPlayer, BBC Red Button and the BBC's digital and mobile services. The blog is reactively moderated. Your host is Nick Reynolds.

Blog Updates

Stay updated with the latest posts from the blog.

Subscribe using:

What are feeds?

Links about BBC Online

BBC Internet blog Archive

owl-plain-112.jpg 2012 ι 2011 ι 2010 ι 2009 ι 2008 ι 2007

Tags for archived posts