Archives for July 2010

What's On BBC Red Button 31st July - 13th August

Post categories:

Lisa Dawson Lisa Dawson | 14:54 UK time, Wednesday, 28 July 2010

Here's our regular look at what's coming up under the red button...

Tracy Beaker Quiz*

Tracy Beaker

Tracy Beaker returns and she's still in trouble! 

Tracy's all grown up now and back at the Dumping Ground but have you been with her through thick and thin? 

Are you a big enough fan to remember what happened when Tracy and Dolly hid 'Fang' the dog in their bedroom?

Join Lisa Coleman, AKA Cam, behind the red button to take the ultimate Tracy test and prove you're the best!

Sky & Virgin:
Monday 9th - Friday 13th August, 7am - 7pm (daily)

Freeview:
Wednesday 11th August, 7am - 2pm & 3pm - 3.50pm
Thursday 12th August, 7am - 4.20pm
Friday 13th August, 7am - 12pm & 1pm - 3.50pm
(Not available on Freesat)

Read the rest of this entry

BBC News website's content management and publishing systems

Post categories:

John O' Donovan | 22:00 UK time, Tuesday, 27 July 2010

As part of the BBC News site refresh we have been making substantial changes to the underlying systems that manage and publish the content.

The BBC has one of the oldest and largest websites on the internet and one of the goals of the update to the News site was to also update some of the core systems that manage content for all our interactive services.

In this post I'll highlight a few areas where we have made some important changes.

CPS

The CPS is the system that manages content production for BBC News, BBC Sport and over 100 other websites across the BBC. It also produces the content for multi-platform journalism, such as the BBC Mobile services and Interactive TV/Red Button services and even content for the mighty Old Skool Ceefax is born in the CPS.

If the BBC website is one of the largest and oldest on the internet, then the CPS has been around nearly as long. As a rule of thumb, if you remember Bagpuss you are older than the CPS. If you grew up watching Teletubbies, you probably are not.

Let's not confuse old with legacy though. The CPS has been constantly evolving and we should say, that when looking at the requirements for the new News site and other services, we did consider whether we should take a trip to the Content Management System (CMS) Showroom and see what shiny new wheels we could get.

However there is an interesting thing about the CPS - most of our users (of which there are over 1,200) think it does a pretty good job [checks inbox for complaints]. Now I'm not saying they have a picture of it next to their kids on the mantelpiece at home, but compared to my experience with many organisations and their CMS, that is something to value highly.

The latest version of the CPS - version 6 - underpins the new News site and has made substantial changes to systems and workflow, but it is still focused on the task of managing content which fits into a general journalistic pattern. It does not try to be all things to all people, and this in no doubt plays some part in its success.

There have been a number of requests from people asking to see more of the CPS but as there is a lot of detail to go into, I'll just focus on a few headline points for now. We will be doing a more in depth blog post on it soon.

Moving to a more structured approach

Some of the major changes in approach are in the Client which is a .NET 3.5 client, taking full advantage of WPF. The screenshot below shows an example image from the CPS which illustrates some new features.

cps-crop.png

This shows a snapshot of a story editing window. Around this are site navigation and other tools (like Search).

As you can see there is a component based structure to the story content with a Video, Introduction and Quote shown. These components are predefined and can be dragged in and added to the story showing that the CPS is not primarily a WYSIWYG editor. The CPS focuses on content structure because in a world where you are publishing to many platforms that have hugely different rendering possibilities WYSIWYG becomes a pointless feature but there are previews showing the output.

Previously, users could add HTML and Custom CPS tags directly into the story body to control the content presentation and the components, similar to the way you would insert code into your content on Wikis and Blogs. This causes a lot of problems for quality and content structure though, so now these things are managed as components where the user can change the content and behaviour of the component in a controlled manner. We will come on to the importance of that next.

HTML Standards

Another part of the CPS that changed considerably was the way content is published. Requirements over the years have caused features to be added organically to the way content is published, leaving it a bit messy with a lot of layout based on HTML tables. A key goal here was to improve the technical quality of content produced and support standards as we move from <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> to <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "https://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

For example, we are aiming for fully valid pages to be published based on the W3C Validation Checker.

valid.png

If you look at some of the older pages published you will see they don't pass this test, and some pages, such as https://news.bbc.co.uk/sport2/hi/olympic_games/default.stm, produce a lot of errors:

validationerrors.png

Ouch!

This is especially tricky to fix where the CPS is pulling in content from other systems or services which don't comply with these standards, but though there is still some work to do here, generally we should be down to 0 or very few errors now.

You will also see in the image above, that older stories are based on the iso-8859-1 character set, whereas new stories will all be UTF-8 encoded for better international support.

Semantic Structure

We will also no longer be using tables to layout the content, instead we will be rendering the pages using CSS layout and only using tables for data.

There are lots of reasons to do this, but some include making the content more efficient, more standards compliant and faster to render. It also allows us to publish semantic XHTML, which means that content blocks are better marked up to describe what they are and has benefits like creating a better header structure to help screen readers.

Better structure also means you will see a more consistent presentation of stories in Google and search engines with, for example, story dates and author information showing more clearly.

This reflects a new content model which is now largely based around a simple and generic data model of assets and groups of assets which are typed (meaning we don't just manage blocks of content, we use metadata to describe what is in the blocks of content) and publishing through templates and services based around Velocity.

Again this is about supporting content standards better as described here for making better use of headings and lists.

Take this example showing how a component is put together.

betterheaderseg.png

Previously the HTML would have looked something like this:

betterheaders01.png

But now it is much more structured and would look something like this with headers clearly marking out the sections of content:

betterheaders02.png

Using CSS for layout also makes a big difference to our HTML and makes for a better separation of layout and content. This rather messy layout...

betterlayout01.png

becomes

betterlayout02.png

The table elements used in the first example are gone and the layout relies on CSS to manage the positioning of content.

URL Structure

Finally a quick note on the change in our URL structure where you may have noticed a couple of significant changes. These are the tip of an iceberg of substantial changes we have made to our networks and infrastructure also part of this relaunch.

The first is that our News URLs have moved from the https://news.bbc.co.uk to https://www.bbc.co.uk/news/ in order to consolidate our domains. As part of the News site changes this involved us making significant updates to our networking infrastructure to allow better sharing of content across our domains. Moving all our URLs onto the https://www.bbc.co.uk/ domain also consolidates some differences which are there largely for reasons no longer necessary.

All URLs should redirect to the appropriate place, but if you do find any broken URLs please let us know.

We also wanted to simplify our URL structure removing much of the baggage in the previous structure for managing different types of content and editions of the website.

Now the structure is basically:

https://www.bbc.co.uk/ [SITE] / [SECTION] - [SUBSECTION] - [STORY-ID]

For example:

https://www.bbc.co.uk/news/uk-politics-10721364

This has made URLS shorter and simpler.

We considered making even shorter URLs - you will have seen some stories were published this way while we transitioned the site to the new design, such as:

https://www.bbc.co.uk/news/10250603

The changes we have made will allow us to make URLs more flexible, and there is more work to do yet on how we might use even shorter URLs (such as https://www.bbc.co.uk/10250603) and longer more descriptive ones https://www.bbc.co.uk/story-about-something-interesting.

If you would like to know more about any of this, then let me know by leaving a comment.

Thanks for reading.

John O'Donovan is Chief Technical Architect, Journalism and Knowledge, BBC Future Media & Technology.

Round up, Monday 26 July 2010

Post categories:

Paul Murphy Paul Murphy | 17:13 UK time, Monday, 26 July 2010

Over on the Radio 4 blog Feedback's Roger Bolton talks to Steve Herrmann about the redesigned news homepage. "...Garish, poorly laid out" says one listener and "I can't believe someone actually designed it to look like this...vile" says another. If you haven't seen it already creative director Paul Sissons blogged about the changes to the BBC News site on the Internet blog and the reasoning behind the redesign.

*
world_music_map_big.jpg

The About the BBC blog has got the details behind the launch of the World Music archive. The archive is on the Radio 3 website and includes an interactive map for those of us whose geography is better than our musical sensibilities.

*

The BBC launched their first mobile app having secured BBC Trust approval following a false start earlier this year. The Independent wrote:

"The BBC Trust has given plans to deliver content through dedicated smartphone applications (apps) the green light, after ruling that they were not a significant change to the BBC's existing public services and did not need further scrutiny."

Not surprisingly the Newspaper Publishers Association who helped trigger the Trust review were disappointed with the outcome. David Newell, director of the NPA, is quoted in a Media Week story:

"The launch of BBC mobile apps represents a significant change to the BBC Online service, and we believe it will have a significant and negative market impact upon the viability of the business models of commercial news organisations in the app market."

On his blog Martin Belam of The Guardian and previously of the BBC wonders if the BBC Trust should have done a Public Value Test before allowing the BBC News app into the Apple store.

You can read the research commissioned by the BBC Trust (and reported by Paidcontent) to help assess whether to give BBC apps the go ahead. The report by Mediatique (PDF is here) says:

"The BBC would be entering a market that is already trending toward free apps (in news, sport and long-form video content) and is likely to trend further in that direction over time, irrespective of the BBC's entry."
*

The Daily Mail has a story on Culture Secretary Jeremy Hunt's appearance on Sunday's Andrew Marr Show:

"Viewers who watch television on their computer could be forced to pay the licence fee as early as next year. Those who do not own a TV but watch programmes on services such as the BBC's iPlayer do not have to pay the £145.50 annual charge."

Paul Murphy is the Editor of the BBC Internet blog.

BBC News mobile apps

Post categories:

David Madden | 11:32 UK time, Friday, 23 July 2010

Today we are launching the BBC News app, initially on the Apple iPhone and iPad and, later in the year, on other platforms as well.

Our aim is to develop a set of core public service apps that bring some of the BBC's most popular, distinctive and original content to mobile in an easy to use and convenient way. The first of these apps is BBC News, and I'll be letting you know about the other BBC apps soon.

So, here's a bit more on the BBC News apps.


BBC News iPhone and iPod Touch app

BBC_News_app_portrait_BBC_copyright.PNG

The BBC News app puts the latest news from our journalists across the UK and the world in the palm of your hand. We've developed an easy to use design which lets you scroll through the latest headlines on your phone. The idea is to create a truly mobile experience that gives you quick access to BBC News on the move.

The app includes the BBC's top Stories, UK and International news, Business, Politics, Health, Education, Science and Environment, Technology and Entertainment stories as well as Features and Analysis from our correspondents around the world.

You can scroll sideways through each news category or up and down the page to access more BBC news sections. To refresh the stories in the app all you need to do is pull down on the interface with you thumb to load the latest news.

We have developed a neat personalisation feature which lets you re-order the news categories to suit your needs. The edit button, top right, reveals a customisation screen where you can add your favourite news sections by clicking on the green + icon and drag and drop each news category into the order of your choice. So, if you like technology news, like I do, you can move that section to the top.

The app includes a latest news banner to keep you up to date with breaking headlines and the BBC News Channel streamed live to your phone so you can keep up with the latest stories from the newsroom.

Clicking on story icons displays the full article on your phone. News stories include embedded video clips and social features so you can share each story via email, Facebook or your Twitter feed. In the article view you can either scroll sideways through each story using the navigation arrows at the top or swipe the story sideways with your thumb to move to the next article.

BBC_News_app_landscape_BBC_copyright.PNG

Finally, we have included a landscape view which you can access by turning your phone sideways. This view provides a handy way to skim through the headlines and get a quick summary of the latest news without having to jump in and out of each article.


BBC News iPad App

BBCapp_ipad.PNG

We have adapted the BBC News iPhone app for the iPad and developed a universal app that works both on the iPhone/iPod Touch and the iPad.

The BBC News iPad app uses the same design as the iPhone app to give you easy access to the latest news. You can scroll through news categories on the left hand side and personalise the experience by adding, removing and re-ordering the news in the Edit menu.

We have included live streaming of the BBC News Channel along with a breaking news banner and social features so you can share stories via email, Facebook and Twitter.

BBCNews_4copyright.PNG

We have also designed a neat portrait view to give a larger reading experience when you turn the device sideways. In this view you can either swipe sideways through articles or click through the story icons at the top of the screen.

We have made a video demo of the BBC News iPad app which gives an overview of how the app works and the features we have developed.

In order to see this content you need to have both Javascript enabled and Flash installed. Visit BBC Webwise for full instructions

This is the first release of the BBC News app in the UK and it will, of course, evolve and improve as we refine the interface and add features. Here's what we are looking at doing next:

  1. Adding BBC Local News so you can get the full range of the BBC's regional coverage on your phone
  2. Adding 3G streaming of the BBC News Channel so you can watch live BBC News when you are out and about. The live stream of the BBC News Channel is currently Wi-Fi only and we've got some BBC infrastructure coming online soon which will enable live streaming over 3G networks.
  3. Further improvements, optimisation and enhancements to make the experience better and quicker

Blackberry Apps

A while ago we developed a series of launcher apps for Blackberry phones to give you quick access to the BBC News, BBC Sport and BBC mobile websites on Blackberry series 4.3 and above phones.

To install these apps on your Blackberry phone all you need to do is type this address into the web browser on your Blackberry phone and click on the icons to install the apps

https://www.bbc.co.uk/mobile/bbapps/

If you have a BlackBerry Bold 2 (9700) or Storm 2 (9520/9550) you can download a short cut app for BBC iPlayer from this page as well.

We will be uploading these apps to Blackberry App World in the coming weeks so you can download them straight to your phone from the Blackberry app store.

The team have worked really hard to design, develop and launch these apps and we would really welcome your comments and feedback. When we Tweet about the apps we are going to use a #bbcapps hashtag so if you would like to use this too that would be great as we are keen to know what you think. Otherwise do post a comment on this blog post.

David Madden is Executive Product Manager, BBC Mobile.

Project Canvas Chairman appointed

Post categories:

Erik Huggers Erik Huggers | 09:43 UK time, Friday, 23 July 2010

Project Canvas' confirmation that Kip Meek has been appointed as Chairman is, I believe, a very important moment for the project I have chaired for the past two years.

Drawing an analogy; for all the partners who have worked together it's like building a new house - at this stage there is much work left to do, but we have laid strong foundations and have now been granted planning permission for the final build. As the first bricks are laid, I wanted to reflect on why the partners wanted to build this house in the first place.

Spurred by the BBC Trust to look at ways of working with other public service broadcasters we established a partnership, initially with ITV and BT, to bring internet services to the television set.

It was clear some years ago that internet-connectivity would have a transformative impact on TV, and the BBC's R&D division - responsible for some of the greatest innovations in broadcasting history - had been looking at how the convergence of broadcast and broadband could work in a single device.

Fast forward to 2008. At this time the BBC and other broadcasters were looking at how audiences could access their VOD offerings on a range of devices - but were faced with a huge problem. From a BBC perspective we already saw a hugely fragmented mobile devices market, with different platforms and operating systems, and the emerging connected-TV space was equally fragmented: Virgin Media, Tiscali Homechoice, BT Vision - all very different proprietary platforms. And while we've already brought BBC iPlayer to Freesat, and soon, Freeview - these existing platforms were conceived as digital broadcasting platforms and for varying reasons are unable to evolve into web-connected platforms by themselves as they stand.

So we looked at three distinct areas. First, it's just not cost-effective to build a bespoke on-demand service and web applications for every different device on the market, which means you have to make tough choices about which devices and platforms you build for. Second, you get a very different user-experience across different platforms and devices since not every feature you see on the web will work elsewhere - which makes it an inconsistent and clunky experience for audiences. Thirdly, as stated in the BBC's sixth public purpose, we have a responsibility to 'deliver to the public the benefit of emerging communications technologies and services and, in addition, taking a leading role in the switchover to digital television.'

Of course it's not just about taking the BBC iPlayer to other platforms - it's all our content - and the approval of Project Canvas won't prevent us syndicating BBC services to new non-canvas platforms and devices, but it is important that choice exists. But for me, there are two main things that will make Canvas distinctive.

First, its focus on simplicity - rooted in the TV experience familiar to everyone in the UK (this was never about putting a search engine on the TV). Technology works best when it's invisible and incredibly simple, useful and fun.

Second, no other platform in the world is working on such open principles with public service at the core. Because the standards are open, the potential audience is huge, and content providers will retain a direct relationship with consumers because the platform has no gatekeepers (editorial or financial). This also means it will become very cost-effective for other services and content providers on the web to build applications for the platform, and Canvas will become a byword for genuine consumer choice.

It was this vision, rooted in public service and developed in a partnership which now comprises ITV, BT, Channel 4, Talk Talk and Arqiva, that has got us to where we are today, and I've been delighted to chair this project throughout its development. As far as the BBC goes, I now need to assess with my team what content and services we make available on the platform and how - but my direct involvement with the team has now finished; and its over to the new Chairman, CEO and Canvas team to realise this vision.

Erik Huggers is Director, BBC Future Media & Technology


BBC News website redesign (5)

Post categories:

Steve Herrmann Steve Herrmann | 19:05 UK time, Wednesday, 21 July 2010

Thank you for all your feedback on the BBC News website redesign. There has been a lot of it, and we'll continue to sort through the comments and e-mails we've been receiving, identifying specific issues we can address and adding answers to the Frequently Asked Questions (FAQs) page.

Most of you commenting here on the Editors blog have been critical, with many urging us to change the design back to the way it was. Given the strength of feeling expressed in some of the comments, I'd like to explain again, as clearly as I can, what our thinking is.

• Reverting to the old design is not something we're considering, but building and continuing to improve on the changes we've made certainly is.
• We are looking closely at the comments and feedback we are getting on all aspects of the new design, and we'll also be carefully watching usage, traffic and conducting further audience research, as we would after any such change.
• There will be a review process and if changes need to be made, they will be considered as part of that.
• There are a few things not yet working exactly the way they should be - our developers and designers are tackling those now, and we are addressing them in the FAQs.
• The changes we have made are based on careful research and thinking about how the site can work at its best now, and how we can make sure it is adaptable enough to continue to evolve, not stand still.

Read more and comment at The Editors at BBC News


Round up: Wednesday 21 July 2010

Post categories:

Nick Reynolds Nick Reynolds | 16:10 UK time, Wednesday, 21 July 2010

Fabrizio Giudici has written a long blog post: "About Android, The Semantic Web and the BBC". Here's an edited extract:

The Semantic Web has been designed exactly to ... have a specific way to integrate data from multiple sources. There are more and more RDF-enabled websites and a very interesting one for my purposes is the BBC, as it is - among other things - a very active producer of nature video footage of excellent quality and known all over the world... Recently, the BBC has done a huge amount of work to make available many pieces of its footage about nature by means of a web interface, the BBC Wildlife Finder (this is part of a larger initiative that makes available also other productions, such as news and sports). Information is both available in HTML for a traditional desktop navigation and in RDF+XML for other uses...

(Thanks to Tom Scott for the link)

Mark Thompson (no relation to the BBC's Director General) reckons: "BBC News iPad app not available in the UK - unjustifiable":

Recently I have been trying to find a good general news app for my shiny new iPad. There are a few out there but what I really wanted was something that provides content of the quality of the BBC News website in an iPad friendly form. Unfortunately I could not find a BBC News iPad app when I looked... Imagine my bemusement then when I discovered today that there actually is such an app and it has been (unsurprisingly) downloaded over a million times globally but that it is not available in the UK. Apparently the BBC Trust is worried that it might hurt the domestic commercial media market.

Comments include contributions from Martin Belam and Mo.

Hugh Garry from Radio 1 is "Rewarding the Love", through "Radio 1's social media channels".

And you've got until Friday to listen to Paul Wakely (Editor, Moderation Services), discussing anonymity on Radio 4's You and Yours (available on iPlayer now - feature starts around 30 minutes in).

Nick Reynolds is Social Media Executive, BBC Online.

BBC iD: Switching off the old sign in system

Post categories:

Nick Reynolds Nick Reynolds | 15:30 UK time, Tuesday, 20 July 2010

Since November of last year we've been upgrading all the BBC's blogs, message boards and communities to a new sign in system: BBC iD.

And we've been getting your feedback along the way. It's been a big task but I think it's gone well. We have now upgraded all the BBC's social media services.

So on Monday of next week, we'll be turning off the old system (called SSO) completely.

If you have an SSO account and haven't upgraded it to a BBC iD as yet then please read on.

sso_bbcid.jpg

You'll still be able to upgrade to BBC iD from an existing SSO account for a few more weeks. However, as a result of turning off the old system, the password reset in SSO will no longer work. From next week on, if you don't know your SSO password then you'll have to create a BBC iD account from scratch.

Most people with active SSO accounts have migrated in the last few months. There are a few services such as Children's and Learning that migrated to BBC iD more recently, whose users are more likely to be affected.

So if you haven't upgraded from SSO to BBC iD then please do so now. And if you know anyone who hasn't upgraded, give them a nudge.

Nick Reynolds is Social Media Executive, BBC Online.

BBC iPlayer press pack: June 2010

Post categories:

Paul Murphy Paul Murphy | 12:10 UK time, Tuesday, 20 July 2010

Ed's note (PM): The BBC iPlayer publicity team have released June's press pack along with some headlines. You can download the pack as a PDF: Publicity_Pack_June_2010.pdf.

  • In total the BBC iPlayer received 117 million requests for programmes across all platforms in June 2010, including both online platforms and devices and BBC iPlayer on Virgin Media TV
  • Other BBC Online sites received significant increases in traffic month on month, as audiences went online for coverage of events and the ability to stream live content from the World Cup, Wimbledon and Glastonbury BBC sites
  • The World Cup also delivered exceptional request numbers for both live and catch-up radio on BBC iPlayer in June, with the England v Slovenia game easily the most requested programme for the month by some margin.
  • Requests from mobile devices also increased another point, boosted by both live World Cup listening, and by the adoption of BBC iPlayer on iPad, which is already delivering 10% of all requests coming from portable devices
  • BBC iPlayer's most-requested TV titles for June continued to include Doctor Who, EastEnders and Junior Apprentice, plus new BBC3 content Lee Nelson, Peckham Finishing School and Mongrels, and of course the England v Slovenia World Cup match.

A new appeals process for the moderation of blogs, message boards and communities

Post categories:

Paul Wakely Paul Wakely | 14:45 UK time, Monday, 19 July 2010

Most people who leave comments on the BBC's blogs, message boards and communities will never get a comment rejected by the moderators, and many of you that do will understand why. But there are times when you need more information so you better understand the rules, and there are times when the mods do get it wrong. So today we are launching a new appeals process for moderation.

When I ran my moderation 'clinics' last year on the PM and BBC Internet blogs and the Points of View message board, I found that many of the same questions about moderation kept cropping up, and some of you also told me that you rarely got an answer if you replied directly to the moderation emails you were sent. So firstly, we had to make sure that you did get replies when you ought to, and secondly that more FAQs would often help you find that information for yourself.

It's taken far longer than I would have liked, but a few weeks ago we introduced a redesigned help section for the BBC message boards with expanded FAQs about moderation and added more information to the BBC blogs help. And today we are introducing a new appeals process for the times when you feel the moderators have got it seriously wrong.

The old system relied on you responding to a moderation email and was devised when we had half a dozen community sites using the DNA moderation system. However, with nearly 300 different blogs, boards, community sites and comments systems now using DNA, it became impossible to even maintain the folders, let alone ensure that all the teams responsible were responding to your moderation queries. Often, the sheer volume of moderation emails - particularly if someone had gone on the rampage with the alert button - made it very hard for teams to find your mails. In time we'd like to find a more elegant way to inform you that your comment has been removed, but for now we had to continue using email.

As well as all the email folders, people who wanted to complain about moderation would write to radio station and TV programme inboxes, post to the boards, phone BBC information or email the BBC Complaints department (who only handle complaints regarding content produced by the BBC, not by the public as your messages and comments are). This all caused duplication and wasted your licence fee.

So from now on, my team will handle all appeals and complaints in the first instance, asking hosts, bloggers or production teams for more information if necessary. The moderation failure emails are shorter and contain a link to more information about the rule your contribution was deemed to have broken. If you wish to appeal you can contact us via the feedback forms on /blogs and /messageboards. You will get an initial response within 10 working days, and if you are unhappy with the outcome, an opportunity to continue with the appeal procedure. If you have restrictions placed on your account, you can also appeal with the new process.

There are some conditions for appeals and we also ask that you make the basis of your appeal clear - general comments about moderation, the BBC, or the state of Britain today will be read but won't result in the decision about your comment being reviewed.  So please say why you consider your post didn't break the rule that it was failed under. Remember the appeal is about your contribution, so "but he was doing it too!" does not constitute an appeal.

For some of our, um, more regular correspondents, we also now have an expedited complaints policy. Sadly, looking at our inbox it seems I also need to point out that any abusive emails will be ignored and might result in a ban if unpleasant enough.

Nobody likes having their comment removed, or their alert rebuffed by the mods, so moderation will never be the most popular aspect of the BBC website. But I hope these changes will help to make moderation clearer, fairer, and more consistent.

Note: I've generally linked to the help material on /blogs here, but all the appeals info is also on the messageboard help pages.

Paul Wakely is Editor, Moderation Services, BBC Online.

A week's work experience with BBC Future Media and Technology

Post categories:

Sariqa Ali | 11:10 UK time, Monday, 19 July 2010

Editor's note (PM): Sariqa Ali spent a week doing work experience in BBC Future Media and Technology having applied via the BBC work experience website. Sariqa has just completed her GCSEs and is planning to take Maths, History, Chemistry and Biology A levels. She's interested in working in the media after college "preferably on the technical or financial side".

This week I was fortunate to spend my time interning at the BBC in the Future Media and Technology department as well as being given the opportunity to blog about my week! So what did I get up to:

Day 1: Social

On the first day, I was with the social team, which was an informative and interesting experience as I got to discover information about their current project, BBC iD, which involves creating an account on the BBC website. I was able to clearly understand what they were discussing as it involved connecting the iD to Facebook and Twitter, two sites I'm extremely familiar with!

I was also given an in-depth insight into the world of a developer, one who helps to actually create the product. I found this information fascinating and intend to continue to research the field.

Day 2: A different aspect of Social

During the first part of the day I learnt about the BBC blogs. I previously ran a blog and it was great to find out how the BBC went about it. I was able to look at the different types of blogs and see what was common between them as well as using the system myself to create a post. I sat in on a meeting as well and was able to witness how members of different departments interacted which gave a great insight into how they came together to discuss a common project and how they worked together.

I also got the chance to visit Television Centre which was exciting, as I was able to see some of the TV studios.

Day 3: Search Editorial Team

I spent the third day with the Search team, initially I didn't realise how much work was put into the search system of the BBC but after spending my day with them I quickly realised the importance of their work creating the unique BBC search system. One of the tasks I received involved finding some children's websites which could be used as external links, which turned out to be pretty challenging. As they were for children they had to be up to a very high certain standard, nevertheless I enjoyed the challenge and managed to come up with a few suggestions.

Day 4: iPlayer and User Experience & Design

Day four was split, in the morning I was given an in-depth insight into the iPlayer and spent time with different members of the team as well as learning about the job of a business analyst. In the afternoon I spent time with the User Experience & Design team. I personally tend to focus on the business and technical side but it was a great opportunity to get into a creative zone. I was given the task of creating my own version of iPlayer and discussed my ideas with the team - who knows, maybe they will appear on the site one day!

Day 5: iPlayer & Marketing, Communication and Audience

My final day was spent with iPlayer. I sat in on a meeting to witness the progress since the previous meeting as well as giving my own feedback regarding the new iPlayer. I was given the opportunity to speak with the Marketing, Communication and Audience team which allowed me to get another perspective of the iPlayer. I enjoyed this as I got to see the way iPlayer makes its way into other forms of media.

Overall the week was a great experience, I was able to find out about many of the different jobs and how a large company works. The teams and staff were incredibly helpful and friendly and made my time at the BBC interesting and memorable.

BBC News website redesign (4)

Post categories:

Steve Herrmann Steve Herrmann | 16:44 UK time, Friday, 16 July 2010

Day three of the redesigned News website, and at the end of a busy week, here's an update on where we've got to.

We've been reviewing all feedback, categorising it, and responding to as much of it as possible, adding the main issues into the Frequently Asked Questions page and summarising them on this blog.

And our journalists have been getting used to the new production tools and different page layouts.

Over the coming days and weeks, we'll be closely watching site traffic data to see how people are actually using the site. We'll also continue to look out for specific issues we can tackle quickly, as we've been doing this week, with updates where relevant to the FAQs page.

Most of the comments in response to my posts here on the redesign have been critical. But, as I've said, we have to assess over time the response of the several million users who come to the site each day by monitoring it in as many ways as we can - this blog is just one of them.

Post-launch reviews are part of any project of this kind: the relevant design, product, technical and editorial people will get together to carefully consider the main areas of feedback and analyse all the data to get as clear a picture as we can of how things are working.

Read the rest of BBC News website redesign (4) on The Editors and leave your comments.

What's On BBC Red Button: 17th - 30th July

Post categories:

John Horth John Horth | 14:47 UK time, Friday, 16 July 2010

Here's our regular look at what's coming up under the red button...

BBC Proms 2010

BBC Proms This Proms season BBC Red Button brings a variety of new ways to watch and listen to this mainstay of the classical music season with a selection of services available alongside the regular broadcasts.

Player Commentary

In a first for BBC Proms this year we're introducing Player Com , an interactive feature giving viewers a new perspective on the Proms from the musicians themselves. By pressing the red button during the performance viewers can listen to live commentary from two or more orchestral players and hear their unique insights into the music as the work is played.

The service will be available for four Proms, starting with commentary for First Night of the Proms from Violinist Nicole Wilson, composer, conductor and choral director Ron Corp, and trumpeter Alistair Mackie.

Beethoven: Maestro Cam & Commentary

Everyone's heard of Beethoven... and now, during the BBC Four Beethoven Piano Concertos on 23rd July, viewers can press the red button to watch Maestro Cam and see BBC Symphony Orchestra chief conductor Jirí Belohlávek bring the master's music to life! We also invite you to stimulate your stave cells with expert commentary on the performance from conductor Matthew Rowe.

And if that weren't enough, Sky and Virgin customers will have the extra option of watching a Solo Cam of one of today's leading Beethoven interpreters, pianist Paul Lewis.

For more on the season visit the BBC Proms website or follow @BBCProms on Twitter

Sky/Virgin/Freeview:
Available during the BBC Four broadcasts

Beethoven Prom
Friday 23rd July, 7.30pm - 9.20pm
Saturday 24th July, 1.50am - 3.40am

Player Commentary
Thursday 29th July, 7.30pm - 9.20pm
Friday 30th July, 2.20am - 4.10am
(Not available on Freesat)

Read the rest of this entry

BBC News website redesign (3)

Post categories:

Steve Herrmann Steve Herrmann | 12:32 UK time, Friday, 16 July 2010

Day two of the new-look BBC News website and we've had lots of feedback - here on this blog at yesterday's post, in messages via our feedback form and on Twitter, Facebook and the wider web.

Taken together, it's a mixture of responses - some pleased, some unhappy and many simply taking note of what's different and getting used to using the site.

There have also been lots of detailed queries, which we're grouping together and answering as many of as we can in this page of Frequently Asked Questions. There is also a new post giving a lot more detail on the design from my colleague Paul Sissons, Creative Director of the project, at the BBC Internet Blog.

Read the rest of BBC News website redesign (3) on The Editors and leave a comment.

BBC News website redesign: telling the story

Post categories:

Paul Sissons | 12:00 UK time, Thursday, 15 July 2010

As you have no doubt noticed, the BBC News website has had a redesign earlier on this week. My colleague Steve Herrmann has posted about this from an editorial perspective; as creative director of the project, I'm going to explain how and why some of the design decisions were made. (See also this picture gallery explaining the new site).

When the News website was launched in 1997, it was only 600 pixels wide.

1998_example.JPG

It had tiny images, no breaking news, no Magazine and certainly no live streaming video. Yet it far exceeded the expectations an early internet user might have had as to how the BBC should deliver news online. Fast-forward 13 years through four major redesigns and we find ourselves in a very different online landscape, with a very different audience which has grown familiar with our design and format.

In the last couple of years, we've seen great leaps in our habits of consumption of content that are influencing web design now. Broadband and smartphone use have changed the speed and convenience with which we're accessing content. Social media services are also playing a part, changing how we communicate, share and manage our online lives.

Our redesign has taken into consideration many of these changes in behaviour, along with our findings from extensive user research and testing, to provide a better template to suit people's evolving needs. It's also a more flexible template which will allow us to implement features in the near future much more easily than in the past.

Navigation

The first thing you might notice is that the left-hand navigation has gone. A horizontal navigation now gives links to the same sections at the top of every page. This was one of the first - and certainly one of the most carefully considered - decisions made on the project, and one that we wouldn't have made without good reason.

First, there was the issue of how much horizontal space the left-hand navigation took up. Story templates and indexes (the front page, the Magazine index and the specialist indexes like business and politics) were always compromised by having 100 or so pixels taken up by a piece of unrelated navigation. Now we can offer larger images and galleries, videos and interactive graphics across the site.

Second, horizontal navigation has long been the standard method of navigation across the rest of the BBC and is an understood interaction that we were sure people would find easy to use.

To support this fundamental design change, we carried out a round of user testing, pitting the current site against one with identical content but a horizontal navigation. Users reacted positively across the board, some not even noticing the difference until it was pointed out to them. With this decision made, we were able to start the redesign of the whole site with a wider, blanker canvas.

Look and feel

Another major improvement to the site is the overall "look and feel" of all pages. We knew that altering the visual design of the site would be changing something familiar and easily used by many. But, as our use of the internet has changed in the past few years, so have our visual expectations.

Technical developments across the web have opened up new avenues for web designers, allowing design with finely-crafted typography for the first time. We worked with UK design luminary Neville Brody at Research Studios on establishing a new "global visual language" (GVL) to establish consistency in design and interaction across all of the BBC website. A more detailed post on this project, the processes used, its overall goals and how it's going to be rolled out across all sites can be read here.

Fundamental to the new GVL is bold, strong type and crisp, un-cluttered layouts. The gradients and textures of "Web 2.0" are gone, and everything is pared down to the minimum required for delivering news.

Words and imagery now sit confidently on the page, with few other distracting elements. Corners are straight-edged, not rounded. Buttons and other interactive elements are consistent and minimal. Everything sits rigidly to a strict baseline grid. Most of these changes might go un-noticed to the non-designers in our audience: we hope that's because they are comfortably finding stories free of distraction.

News front page and sections

Designing a news site's front page is a huge challenge, with dozens of internal and external stakeholders. It is the "shop front" through which many people start their journey and everyone has an opinion as to what should be in the window displays.

The BBC news website must highlight stories on a wide variety of subjects. The TV news output of a given week ranges from Panorama to Breakfast, and we have to present content of similar range on the site daily. We could have tried to satisfy everyone by giving each section an equal amount of permanent space, but that would make the site look far too much like a patchwork of wildly differing subjects. Instead, we developed a system to control the "volume" of subjects and stories.

In the previous design, the top story always carried the biggest image, with stories two and three taking smaller image sizes. This did not always work well if, for example, there were no strong images associated with the top story but, say, the number three story potentially had a great picture to go with it. We needed to create a template that was flexible enough to allow us to pick the right image size for the story, without being tied completely to its place on the page.

So now each of the top stories can be shown at different "volumes". A featured story can appear larger, with a picture or not, regardless of its place in the list of top stories. This allows a much broader scope for the index editor, who makes the decisions around what's being included on a day-to-day basis. For instance, a Glastonbury story can be shown with a large image, while a more newsworthy Budget story unfolds above it. The Budget remains the most visually prominent, and therefore the most editorially significant by its use of large, bold type.

These new flexible modules also bring more flexibility to section indexes. For example, the entertainment index can use image-based presentation for more of its stories, and the business index can use it for fewer. Images no longer need to be used just for the sake of it. This also frees up time where previously images needed to be sourced and cropped.

volume_examples.JPG

Another insight from user research was about scrolling habits. We'd assumed that nobody likes to scroll much on the web. After all, a rule common to most websites and web designers is to keep key content above the "fold". But we don't believe that's the case any more.

For instance, statistics show that a large amount of users use the "most popular" panel on story pages as their main way of moving around the site. Yet they weren't spending long enough on the pages to be actually reading more than the first few paragraphs. Instead, they were willing to scroll to a piece of navigation they were comfortable with.

With an incentive, users will scroll. If that proves a positive interaction, it's something that could become habitual. So rather than design our indexes and front page with everything at the top of the page, we are encouraging scrolling by putting richer content within stories and towards the bottom. In all our previous index designs, content became increasingly sparse as you scrolled. Large images appeared up top, but none were visible once they'd scrolled out of view. For the redesign, we've developed a consistent visually-rich "digest" that sits at the bottom of indexes.

Now a user can browse the main headlines, then scroll down to see many more from around the site. Rather than appearing as a footnote, they're given the same visual prominence as the main news of the day.

Story pages

Our previous template for articles was much loved, but could be improved. We know from user research that people are happy with the overall structure of a news article page, but analytics show us that the layout could be more effective.

Though the removal of the left-hand navigation gave us more horizontal space, we were apprehensive about using this as an opportunity to widen the story body. The current layout offers a comfortable reading experience with optimum type size and line length.

When we refreshed the site in 2008, some feedback suggested that the site's width should be stretched (flexible layout) and the user's browser size should set the line length for each individual. This has been a debate between web designers for many years. There are benefits and disadvantages to both approaches. Due to the complexity and unpredictability of page layouts, we have opted for a fixed layout so we can be sure everyone is seeing one consistent design

When our journalists create stories, they place images, fact-boxes and video and audio with an understanding of where they will appear to the user. To offer a flexible layout would lose that control, creating unpredictable layouts that wouldn't look great. Users can still view stories at a flexible width using an RSS reader, but for the "hi-web" version, stories need to be presented as well as possible.

The additional horizontal space does, though, free up space for bigger images and embedded videos. Special stories with interactive graphics can now sit across the full page width, allowing for richer infographics. And when the new space doesn't have a good reason to be filled with content, it remains blank, allowing the story to breathe.

Across the project, there was a concern that larger images, bolder headlines and more links could make pages overbearing. Having a column of white at the right-hand side of a story's body was one of the design decisions made explicitly to counter this.

A major change to the layout of story pages is the new placement of related stories and unrelated ones. In the previous story templates, links related to the story you're reading sit close-by in the top right. But our analytics show that not a lot of people are using those links. They act as a 'foothold', a familiar place of reference to read headlines of similar or earlier stories on the subject. But you can achieve that in a lot less space than we're currently using, with far fewer items. So the related links visible at the top of the page are now limited to four, and are embedded in the story body. For those who want to read in depth on a subject related to the article, the full offering of all related content now sits at the bottom, after the main story body.

Replacing the related links in the top right is a selection of modules promoting stories across the site. Around a quarter of you every day are now arriving at BBC News directly into story pages, after following a link from another site, such as a social network or news aggregation site. This means that many people are visiting stories, but not getting to see the main news of the day. With Top Stories prominently visible to every user, we allow for more 'sideways navigation'. People won't have to click back and forth from Front Page to story in order to read the stories of the day.

There are further modules beneath Top Stories, promoting the best Features & Analysis around the site, along with the ever-popular 'Most Popular' panel. In fact, it worked so well in the previous design you might ask why we didn't just move it to the top of this column.

If it's what people use and like, why not make it most prominent?

Well, it often consists of the quirkier stories on the site and is a varied mix. But people are finding this module easily anyway, and we wanted to still be able to communicate a sense of our own editorial priorities near the top of the page by picking out the key top stories of the day there.

Video

The integration of video content has been evolving almost as long as the site itself. Initially only very short clips were available, and at a very low quality. Their relationship to written stories was as additional, optional content. Over the years their quality has grown, as has their length. Now, thanks to the amazing work by the iPlayer team, we have an established format for long-form, high-quality programmes across the BBC.

But our integration of this content into News still had the legacy of it being added to the original structure of story pages. Video clips were embedded in story pages, or could be viewed as what we call "media asset pages" (MAPs), but these hadn't been created based on how people were using video or what they wanted from it. Now the MAPs have been purpose-designed to allow a richer media experience, using some of the interaction patterns from iPlayer.

Many people enjoyed navigating sideways between videos using the "most popular" panel, so we've made more of this. 15 videos are now browsable in the vertical carousel that sits at to the right of each clip. Additionally, we used our insight about users not necessarily being averse to scrolling again. If the user is viewing video content, and wants to watch more, he or she can scroll down to direct links to dozens of them, presented with images and divided by subject matter. And of course, a major aspect of these redesigned video pages is that the video itself is now significantly bigger.

Adverts

While the UK news site is licence-fee-funded, the international version is funded by advertising via our commercial arm, BBC Worldwide. Through investment from BBC Worldwide, we were set the dual challenge of creating an ad-free UK site layout which can seamlessly integrate and showcase advertising for the international audience. There aren't many other sites that have to produce content that appears so differently to people depending on their location.

One ad format proved difficult to implement, as its size would affect the whole structure and gridded layout of the site. So we devised a flexible layout, which stretches the right-hand column when a large ad is served. Like many of the improvements, it's something most of the audience will barely notice, but it was an essential problem to solve in order to offer the international version of the site to advertisers as a competitive commercial product.

The future

We have more exciting things in the pipeline. This template allows us to incorporate them easily, and to continue to adapt working alongside our audience. If you have any further questions about our design processes or why certain decisions have been made, feel free to ask them below and we'll try to answer them. Please bear in mind, though, that what you see today is just the first step.

Paul Sissons is UX Team Lead, BBC Future Media & Technology

Zeitgeist - the most shared BBC links on Twitter

Post categories:

Nick Reynolds Nick Reynolds | 08:58 UK time, Thursday, 15 July 2010

Zeitgeist is a prototype developed by our colleagues in BBC Research and Development.

It's aim:

...to highlight the most shared BBC webpages on Twitter, a digest to link people to the hottest BBC pages

On the R&D blog Theo Jones explains more:

We use the Twitter streaming API to access the Gardenhose sample stream, which provides a subset of the full Twitter message stream, at a rate of about 100 messages per second and to track "BBC" as a keyword. These messages are then fed into a pipeline of processes written in Ruby connected by queues provided by RabbitMQ, a fast and reliable messaging server.

Read more and comment at the BBC Research and Development blog.

Nick Reynolds is Social Media Executive, BBC Online

BBC News website redesign (2)

Post categories:

Steve Herrmann Steve Herrmann | 08:05 UK time, Wednesday, 14 July 2010

Welcome to the new-look BBC News website. As previewed last week on this blog, we're introducing a number of improvements from today to our design and layout.

The full range of content is still all here - the best of the BBC's journalism in text, video, audio and graphics - but we've set out to make it easier for you to find, use and share. In summary, and to recap on my earlier post which gives more details:

What's new:
• a fresh, updated design, with more space for the main stories of the day
• better use of video and images
• clearer and more prominent labelling and signposting of key stories, whether you are on the front page or a story page
• a better indication of which are the most recent headlines
• easier ways to share stories with others, for those who wish to, on social media networks

To read the rest of BBC News website redesign (2) and to comment on the new-look BBC News website go to The Editors.

BBC News website redesign

Post categories:

Erik Huggers Erik Huggers | 07:34 UK time, Wednesday, 14 July 2010

Today sees the launch of the redesigned BBC News website. This is the first major part of the BBC website to have implemented our new online design guidelines, known as global visual language. My colleague Steve Herrmann in News has blogged in detail today about the improvements - and I hope you'll like them - but I wanted to reflect on why this moment isn't just important to the News website but to BBC Online as a whole.

This is part of an ongoing process to make BBC Online feel like one coherent service, rather than a disjointed collection of websites, which is greater than the sum of its parts.

Our aim is more than making a website easy on the eye; a good user-experience is essential to making the site easy to use, and most importantly to make it easy to find what you want to look at quickly.

Read more and comment at the About the BBC blog

Erik Huggers is Director, BBC Future Media & Technology

Formative user research for the BBC's Global Visual Language 3

Following Bron's post on the new Global Visual Language we received several questions about our approach to user research. Hopefully we can shed some light on some of our activities.

We began our research with a literature review and then performed a competitor analysis to identify our strengths, weaknesses and opportunities This was followed-up with a series of in-depth interviews, surveys, card sorts and diary studies; and finally usability testing on early paper prototypes and high-fidelity prototypes.

The studies were carried out both in-house and with external usability agencies, and provided us with heaps of qualitative data.

The main objectives at this stage were:

  • To uncover users' natural internet behaviour; and more specifically people's use of the BBC digital services on different devices (e.g. PC, mobile)
  • To validate the GVL3 design philosophy and gain insights into how users interpret the set of values based on their experience with the BBC digital services
  • To explore the concept of 'modern' web design
  • To find out how users recognise websites they are familiar with and what design elements trigger recognition of online brands

The last objective led to a rather interesting activity, the 'onion-skinning' exercise, which was based on the theory of visual recognition (as opposed to recall) memory. In short, the theory says that objects (eg people's faces) stick in our recognition memory by experience and with frequent exposure to them. To recognise something or someone we look for so called visual cues (e.g. shapes, colours) which help us spot the objects we are familiar with among others we have not seen before (Baddeley, 2009). In turn a visual recognition memory task provides visual cues that facilitate searching through memory for help in recognising familiar objects.

In our 'onion-skinning' exercise we selected the following six design elements (or visual cues): layout, header & footer, icons, typeface, images and colours. We then selected ten popular websites and deconstructed each of them into the six cues.

For this particular study we recruited fourteen participants aged 16 to 55+ (ensuring a 50/50 gender split, and a mix of experienced and inexperienced internet users). It was crucial that all fourteen participants were familiar with the BBC Homepage and regularly used at least five of the following websites: BBC News, BBC iPlayer, Apple store, BT, eBay, Guardian, MSN, Wikipedia and Yahoo.

We started each session with a conversation about how important the design of websites was for participants. It was quite clear from the beginning that the participants naturally fell into one of two groups: 'design-aware' participants for whom design was very important and could potentially either attract them of put them off using websites, and another group of 'content-driven' participants, who weren't so much concerned with the design as long as they were able to easily find information they wanted. Perhaps unsurprisingly, the division seemed related to age with the younger participants being more design-aware.

The participants were presented with six web pages they had said they were familiar with. They saw each page in six steps (with new cues added each time).

After they saw each cue, the participants were asked whether they recognised the page, how confident they were, and to describe what had helped them recognise the brand.

Here's an example. The first cue was layout.

2a_595.jpg

Do you recognise the page shown above?

In the study the only page that was recognised at this first stage was a Wikipedia article page (but even then only two out of fourteen participants). The minimal design and consistency of layout seemed to help users with recognition. Both participants were in the 'design-aware' group.

It was also interesting to observe that participants had clear expectations about how certain type of pages were laid out (such as a news page, a social networking page or media player page).

The next layer included icons (in black and white):

2b_595.jpg

Icons appeared to play a rather important role in triggering recognition of websites when used sporadically and when very distinct (the rosette 'top-rated seller' on eBay, or the home icon on the Apple Store page). If used consistently across products and services, icons would play an important role in recognising a brand.

The next layer introduced the header, footer and typography (with dummy text).

2e_595.jpg

Have you recognised it as a BBC News article page yet?

The typeface was one of the features that triggered the recognition of pages for most participants. Comments on the BBC News page and BBC homepage were: "Now I'm confident it is a BBC page, this is the BBC font." (Male aged 16) and "BBC-like font, it's quite identifiable." (Female aged 19). It was interesting to observe that the font we were using was so strongly associated with the BBC. However, quite a few participants suggested that this was a time for change: "I grew up with it [the same font]. It's time for change. It needs something fresher" (Male aged 48)

The suggested change of typeface tallied with out own ambition and we took it seriously. Participants expected the new BBC font to be neat and modern-looking.

The final step was colour.

2g_595.jpg

Are you confident now? Images and colours definitely helped the participants recognise pages. The more 'content-driven' participants in particular had to go through all six stages of the exercise until they were absolutely sure. The strongest elements on current BBC websites were: the typeface, images and colour (e.g. red on BBC news article page).

This is an example of just one exercise we carried out in the formative stage of our research. The findings helped us realise that users were very perceptive to typeface, icons and colours; and that consistency and simplicity of these elements would ensure that we were providing our users with coherent, joined-up experience across the site.

Sylwia Frankowska-Takhari is Usability & Accessibility Specialist, FM&T User Experience & Design.

References: Baddeley, A., Eysenck, M.W., Anderson, M.C. (2009). Memory. Hove, UK.: Psychology Press.

BBC R&D Recruiting: Lead Technologist (Audio)

Post categories:

Paul Murphy Paul Murphy | 14:44 UK time, Tuesday, 13 July 2010

The BBC R&D blog have put out a call for audio technologists. Here are the details:

"As a part of its Salford-based lab being established at MediaCityUK, BBC R&D wishes to recruit a Lead Technologist to lead the work of the audio R&D engineering team that will be based in this lab.

Lead Technologist (Audio)

This is an exciting opportunity for someone with a background in audio R&D and a proven track record in leading R&D teams to play a key role in the development of the new lab and to shape the BBC's future audio R&D work. The initial areas of research are likely to include periphony, spatial audio and Ambisonics, and the related areas of room acoustics, but could expand to include any aspects of media-related audio R&D.

mediacityUK_HDR_Caza_No_7_500.jpg

CC licensed image of MediacityUK by Ian Carroll

As the BBC R&D team in our North lab builds out it's capabilities and facilities we need a world class technical leader to focus our audio work.  This role is critical to developing the excellence in audio we want at the heart of our operation, and it'll sit right in the nexus of industrial and academic partnerships that will span the region, the UK and the wider industry."

See the original post on the R&D blog.



BBC iPlayer on Android update

Post categories:

David Madden | 16:35 UK time, Monday, 12 July 2010

I've been following the comments on my recent post about BBC iPlayer on mobile on Android 2.2 phones with interest and want to address some of the points raised.

First, it's worth reflecting on what we are trying to achieve with the BBC iPlayer on mobile service. The BBC Online Service Licence, issued in May 2010, describes iPlayer's objectives as:

"BBC iPlayer should enable licence fee payers to access BBC programming quickly, easily and in a high quality format. In doing so, it should aim to be regarded as a high quality BBC service by its users and so contribute to their approval of the BBC.

BBC iPlayer should aim to maintain the BBC's overall reach and consumption levels, as usage of the BBC's linear services is replaced over time by on-demand consumption. In doing so, it should contribute in the long term to the BBC's ambition to provide services that are of value to all licence fee payers. It should aim at least to maintain consumption of BBC content by younger adults (those aged 16-34)....

...In fulfilling its other aims and objectives, BBC iPlayer should aim to contribute to the growth in the usage of rich media in broadband households. Within a reasonable timescale, it should aim to make the seven-day catch-up offering available on a platform-neutral basis, or at the least to be available on all major platforms subject to value for money considerations and as technology allows."

Given these overall objectives, BBC iPlayer on mobile is tasked with maximising reach on mobile platforms while delivering a high quality BBC service in a cost-effective way.

The big question, and it's a question being pondered by other content providers right across the industry, is: how do we scale services across multiple mobile platforms in a cost and resource efficient way?

The mobile landscape is very fragmented with a host of operating systems and a proliferation of screen sizes, resolutions, video codecs and web browsers. Developing for each platform soon becomes very expensive. Maintaining and supporting each variant requires more and more resource as each operating system releases new firmware versions and upgrades.

Rolling out new BBC iPlayer features across all mobile platforms would also be increasingly expensive as would the associated testing and support effort. As new phones and new operating systems enter the market, we would be obliged to support the new ones as well as the old ones adding to our support overheads.

To get an idea of the range of mobile platforms and the potential complexity of development see, for example, this Wikipedia article about mobile application development.

Given these development challenges, our approach has been to build a scalable website that works in the phone's web browser and can be easily tweaked to achieve that high quality experience on a range of internet enabled mobile devices.

The BBC iPlayer on mobile website is modular with a series of components that can be easily switched on or off depending on the phone's capabilities. The practical upshot of this is that if there's a feature which your phone can't handle, you won't even see it (rather than having something that's there even if it won't work for you).

The advantages of a web solution for BBC iPlayer on mobile is we can leverage the BBC's existing web technologies and software development skills while minimising the number of iPlayer variants and special builds we have to support. We just have to build and maintain a single website.

Our web approach also means that new features, like those rolled out with the recent Version 3 release, only need to be built once, rather than for each variant or operating system. We also benefit from infrastructure efficiencies by using existing servers, development environments and encoding and delivery systems. For more on the infrastructure behind BBC iPlayer see Marina Kalkanis's blog post.

We could have enabled the BBC iPlayer on mobile website on all video enabled phones without any restrictions or exceptions. This would have maximized our reach, but would have resulted in a very poor quality experience on many phones as video playback capabilities and web browser rendering vary across devices. Some users would have had a good experience while others suffered a sub-optimal service with features not working and poor video playback quality.

To maintain the consistently high quality service demanded by our service licence we have had to test the BBC iPlayer on mobile website on a device-by-device basis to make sure that everything works and we deliver the best possible user experience.

So, how do we prioritise which phones we test and enable BBC iPlayer on?

We look at the reach potential of a device to understand how many licence fee payers we can make the service available to through that phone. We also evaluate the resource and maintenance costs of enabling a high quality iPlayer experience on that device. In addition we assess whether we can apply technology solutions we already have to new devices with minimum effort, an example of this would be BBC iPlayer streaming on iPad as the tech needed is very similar to that which enables us to stream iPlayer on iPhones. This is driven by our overall objective of maximizing reach on mobile platforms while delivering a high quality BBC service in a cost-effective way.

We have limited resources on BBC iPlayer on mobile and therefore have to carefully prioritise development work to maximise reach and value. So, if, for example, I have 15 units of work I need to do on mobile iPlayer (support, maintenance, new features, new handsets etc) but only 5 units of effort available, I've got to focus on the high volume phones to get the service out to as many people as possible.

I hope that gives an overview of what the BBC is trying to achieve with BBC iPlayer on mobile and outlines the approach we have adopted.

I'd now like to turn to some of the specific questions raised in the comments on my previous blog post.

Tiggs questioned why the BBC took down the beebPlayer which worked on older Android devices and did not rely on Flash, and why we have replaced it with something that only works on newer devices and requires Flash.

The BBC's syndication policy, which governs how the BBC makes its services available through other parties, clearly outlines the criteria for using BBC content. BeebPlayer was not a licensed distributor of BBC content online or on mobile. The BBC routinely looks for unauthorised usage of our brand and our content across all platforms and when we encounter it we work to resolve the issue. If on investigation we find that a company's service proposition does not adhere to our standard licence terms and conditions, we will take steps to remedy the issue.

Why has the BBC replaced beebPlayer with something that only works on newer devices and requires Flash?

Using Adobe Flash 10.1 streaming on mobile delivers significant infrastructure efficiencies for the BBC, as we use our existing video and audio encoding plant to create the streams. We don't need to install any new kit or set up any new servers. We just use what we already have to offer a higher quality BBC iPlayer on mobile experience.

Enabling Flash on Android 2.2 devices also means that all current and new devices that support Android 2.2 can get BBC iPlayer. These devices all use the same standard Flash player which means we can offer a consistently high quality playback across all of them. Previously we had to review and test BBC iPlayer on a device-by-device basis to ensure the right high quality experience. Now we can offer BBC iPlayer on mobile to a whole group of devices at once, which is clearly much more efficient.

Chris questioned why the BBC has chosen Flash over a more open and accessible standard.

Adobe Flash reaches an estimated 95% of PCs which means the BBC can use Flash streaming technologies to reach audiences on the internet right across the UK with a consistent video playback experience.

As soon as Flash streaming came to mobile, through Adobe's Flash 10.1 player on Android 2.2 devices, it made sense to make the most of our existing Flash infrastructure to bring that consistent playback experience to mobile as well.

Why haven't we enabled BBC iPlayer on mobile on any other Android phones apart from Android version 2.2?

BBC iPlayer on mobile's reach objectives mean we have had to prioritise other devices that offer the BBC wider reach over current Android phones.

The best way to bring BBC iPlayer to earlier versions of Android (which don't support Flash), is to develop an app. This would provide a single scalable version that could be offered to all Android phones.

The BBC Trust is conducting a review of the BBC's plans to develop smartphone apps. The BBC will therefore not be launching any Android apps or apps for any other smartphone in the UK pending the outcome of the BBC Trust review.

David Madden is Executive Product Manager for BBC iPlayer on Mobile.

BBC World Cup 2010 dynamic semantic publishing

Post categories:

Jem Rayfield | 10:00 UK time, Monday, 12 July 2010

The World Cup 2010 website is a significant step change in the way that content is published. From first using the site, the most striking changes are the horizontal navigation and the larger, format high-quality video. As you navigate through the site it becomes apparent that this is a far deeper and richer use of content than can be achieved through traditional CMS-driven publishing solutions.

The site features 700-plus team, group and player pages, which are powered by a high-performance dynamic semantic publishing framework. This framework facilitates the publication of automated metadata-driven web pages that are light-touch, requiring minimal journalistic management, as they automatically aggregate and render links to relevant stories.

eng_595.jpg

Dynamic aggregation examples include:

The underlying publishing framework does not author content directly; rather it publishes data about the content - metadata. The published metadata describes the world cup content at a fairly low-level of granularity, providing rich content relationships and semantic navigation. By querying this published metadata we are able to create dynamic page aggregations for teams, groups and players.

The foundation of these dynamic aggregations is a rich ontological domain model. The ontology describes entity existence, groups and relationships between the things/concepts that describe the World Cup. For example, "Frank Lampard" is part of the "England Squad" and the "England Squad" competes in "Group C" of the "FIFA World Cup 2010".

The ontology also describes journalist-authored assets (stories, blogs, profiles, images, video and statistics) and enables them to be associated to concepts within the domain model. Thus a story with an "England Squad" concept relationship provides the basis for a dynamic query aggregation for the England Squad page "All stories tagged with England Squad".

This diagram gives a high-level overview of the main architectural components of this domain-driven, dynamic rendering framework.

diagram_595.png

The journalists use a web tool, called 'Graffiti', for the selective association - or tagging - of concepts to content. For example, a journalist may associate the concept "Frank Lampard" with the story "Goal re-ignites technology row".

In addition to the manual selective tagging process, journalist-authored content is automatically analysed against the World Cup ontology. A natural language and ontological determiner process automatically extracts World Cup concepts embedded within a textual representation of a story. The concepts are moderated and, again, selectively applied before publication. Moderated, automated concept analysis improves the depth, breadth and quality of metadata publishing.

Journalist-published metadata is captured and made persistent for querying using the resource description framework (RDF) metadata representation and triple store technology. A RDF triplestore and SPARQL approach was chosen over and above traditional relational database technologies due to the requirements for interpretation of metadata with respect to an ontological domain model. The high level goal is that the domain ontology allows for intelligent mapping of journalist assets to concepts and queries. The chosen triplestore provides reasoning following the forward-chaining model and thus implied inferred statements are automatically derived from the explicitly applied journalist metadata concepts. For example, if a journalist selects and applies the single concept "Frank Lampard", then the framework infers and applies concepts such as "England Squad", "Group C" and "FIFA World Cup 2010" (as generated triples within the triple store).

This inference capability makes both the journalist tagging and the triple store powered SPARQL queries simpler and indeed quicker than a traditional SQL approach. Dynamic aggregations based on inferred statements increase the quality and breadth of content across the site. The RDF triple approach also facilitates agile modeling, whereas traditional relational schema modeling is less flexible and also increases query complexity.


Our triple store is deployed multi-data center in a resilient, clustered, performant and horizontally scalable fashion, allowing future expansion for additional ontologies and indeed linked open data (LOD) sets.

The triple store is abstracted via a JAVA/Spring/CXF JSR 311 compliant REST service. The REST API is accessible via HTTPs with an appropriate certificate. The API is designed as a generic façade onto the triplestore allowing RDF data to be re-purposed and re-used pan BBC. This service orchestrates SPARQL queries and ensures that results are dynamically cached with a low 'time-to-live' (TTL) (1 minute) expiry cross data center using memcached.

All RDF metadata transactions sent to the API for CRUD operations are validated against associated ontologies before any persistence operations are invoked. This validation process ensures that RDF conforms to underlying ontologies and ensures data consistency. The validation libraries used include Jena Eyeball. The API also performs content transformations between the various flavors of RDF such as N3 or XML RDF. Example RDF views on the data include:

Automated XML sports stats feeds from various sources are delivered and processed by the BBC. These feeds are now also transformed into an RDF representation. The transformation process maps feed supplier ids onto corresponding ontology concepts and thus aligns external provider data with the RDF ontology representation with the triple store. Sports stats for matches, teams and players are aggregated inline and served dynamically from the persistent triple store.

The following "Frank Lampard" player page includes dynamic sports stats data served via SPARQL queries from the persistent triple store:

frank_595.jpg


The dynamic aggregation and publishing page-rendering layer is built using a Zend PHP and memcached stack. The PHP layer requests an RDF representation of a particular concept or concepts from the REST service layer based on the audience's URL request. If an "England Squad" page request is received by the PHP code several RDF queries will be invoked over HTTPs to the REST service layer below.

The render layer will then dynamically aggregate several asset types (stories, blogs, feeds, images, profiles and statistics) for a particular concept such as "England Squad". The resultant view and RDF is cached with a low TTL (1 minute) at the render layer for subsequent requests from the audience. The PHP layer dynamically renders views based on HTTP headers providing content negotiated HTML and/or RDF for each and every page.

To make use of the significant number of existing static news kit and architecture (apache servers, HTTP load balancers and gateway architecture) all HTTP responses are annotated with appropriate low (1 minute) cache expires headers. This HTTP caching increases the scalability of the platform and also allows content delivery network caching (CDN) if demand requires.

This dynamic semantic publishing architecture has been serving millions of page requests a day throughout the World Cup with continually changing OWL reasoned semantic RDF data. The platform currently serves an average of a million SPARQL queries a day with a peak RDF transaction rate of 100s of player statistics per minute. Cache expiry at all layers within the framework is 1 minute proving a dynamic, rapidly changing domain and statistic-driven user experience.

The development of this new high-performance dynamic semantic publishing stack is a great innovation for the BBC as we are the first to use this technology on such a high-profile site. It also puts us at the cutting edge of development for the next phase of the Internet, Web 3.0.

So what's next for the platform after the World Cup? There are many engaging expansion possibilities: such as extending the World Cup approach throughout the sport site; making BBC assets geographically 'aware' is another possibility; as is aligning news stories to BBC programs. This is all still to be decided, but one thing we are certain of is that this technological approach will play a key role in the creation, navigation and management of over 12,000 athletes and index pages for the London 2012 Olympics.


Jem Rayfield is Senior Technical Architect, BBC News and Knowledge. Read the previous post on the Internet blog that covers the BBC World Cup website, The World Cup and a call to action around Linked Data.


Metadata is data about data - it describes other data. In this instance, it provides information about the content of a digital asset. For example, a World Cup story may include metadata that describes which football players are mentioned within the text of a story. The metadata may also describe the associated team, group or organization associated to the story.

IBM LanguageWare Language and ontological linguistic platform.

RDF is based upon the idea of making statements about concepts/resources in the form of subject-predicate-object expressions. These expressions are known as triples in RDF terminology. The subject denotes the resource; and the predicate denotes traits or aspects of the resource and expresses a relationship between the subject and the object. For example, to represent the notion "Frank Lampard plays for England" in RDF is as a triple, the subject is "Frank Lampard"; the predicate is "plays for" and the object is "England Squad".

SPARQL (pronounced "sparkle") is an RDF query language its name is a recursive acronym (i.e. an acronym that refers to itself) that stands for SPARQL Protocol and RDF Query Language.

BigOWLIM A high performance, scalable, resilient triplestore with robust OWL reasoning support

LOD The term Linked Open Data is used to describe a method of exposing, sharing, and connecting data via dereferenceable URIs on the Web.

JAVA Object-orientated programming language developed by Sun Microsystems.

Spring Rich JAVA framework for managing POJOs providing facilities such as inversion of control (ioc) and aspect orientated programming

Apache CXF JAVA Web services framework for JAX-WS and JAR-RS

JSR 311 Java standard specification API for RESTful web services.

Memcached Distributed memory caching system (deployed multi datacenter)

Jena Eyeball JAVA RDF validation library for checking ontological issues with RDF

N3 Shorthand textual representation of RDF designed with human readability in mind.

XML RDF XML representation of an RDF graph.

XML (Extensible Markup Language) is a set of rules for encoding documents and data in machine-readable form

Zend Open source scripting virtual machine for PHP, facilitating common programming patterns such as model view controller.

PHP Hypertext Preprocessor general-purpose dynamic web scripting language, use to create dynamic web pages.

CDN A content delivery network or content distribution network (CDN) is a collection of computers usually hosted within Internet Service Provider hosting facilities. The CDN servers cache local copies of content to maximize bandwidth and reduce requests to origin servers.

OWL Web Ontology Language (OWL) is a family of knowledge representation languages for authoring ontologies.

The World Cup and a call to action around Linked Data

Post categories:

John O' Donovan | 13:37 UK time, Friday, 9 July 2010

Underneath the surface of the BBC World Cup web site there is a revolution going on in the technology and workflow being used to manage and publish our content. To some extent we have been doing this in stealth mode as we figure out a lot of challenges, but as we approach the World Cup Final we'd like to explain some more about what we have changed and why this is an important engagement for us in the development of the Semantic Web and support for the use of Linked Data .

For some time, we have been working on utilising Metadata and Linked Data to organise and manage the site dynamically, culminating in the World Cup 2010 site which uses Linked Data to manage how content is published. We have also had some News Linked Data discussions with other news organisations thinking about how to bring a critical mass to the development of the Semantic Web and what benefits it can bring.

The World Cup site is our first major statement on how we think this can work for mass market media and a showcase for the benefits it brings.

First some background on the World Cup site.

The World Cup site is a large site with over 700 aggregation pages (called index pages) designed to lead you on to the thousands of story pages and content which make up the whole site. Examples of index pages range from the Groups and Fixtures page through to detailed pages for each team or player.

Normally, managing all these index pages for the World Cup would not be possible as each of these needs to be curated by an editor, setting up automation rules or keeping it up to date with latest stories and information. To put the scale of this task in perspective, the World Cup site has more index pages than the rest of the Sport site!

So how is this possible? Clearly some form of automation is required, but search technologies and previous methods for doing this have proven to be inaccurate and there is no point in having all these pages if the quality of them is perceived to be low. You don't want to get content mixed up between different players with the same surname, for example.

The key change is we are using some advanced methods for analysing content and deciding how to tag this content with precise metadata linked to uniquely identified concepts (a concept usually being a person, place or thing). In the case of the world cup we are interested in players, teams, matches, etc... but the principle can be easily applied to anything. To do this we are using some technology from IBM (Languageware) and Ontotext (BigOWLIM) and a high level view of the process is shown in Fig 1, but we will be following up this post with more details about how this all works.

meta_process_world_cup.jpg

Pushing the Boundaries

Though there are lots of dynamically published sites on the internet, the difference here is in the use of RDF and Linked Data to build and manage the site. This is incredibly flexible and we are only just starting to explore the possibilities of how this allows us to present and share content. Though we have been using RDF and linked data on some other sites (such as BBC Programmes, BBC Wildlife finder, Winter Olympics) we believe this is the first large scale, mass media site to be using concept extraction, RDF and a Triple store to deliver content.

Another way to think about all this, is that we are not publishing pages, but publishing content as assets which are then organised by the metadata dynamically into pages, but could be re-organised into any format we want much more easily than we could before.

world_cup_publishing.jpg

So why is this important?

The principles behind this are the ones at the foundation of the next phase of the internet, sometimes called the Semantic Web, sometimes called Web 3.0. The goal is to be able to more easily and accurately aggregate content, find it and share it across many sources. From these simple relationships and building blocks you can dynamically build up incredibly rich sites and navigation on any platform.

There is also a change in editorial workflow for creating content and managing the site. This changes from publishing stories and index pages, to one where you publish content and check the suggested tags are correct. The index pages are published automatically. This process is what assures us of the highest quality output, but still saves large amounts of time in managing the site and makes it possible for us to efficiently run so many pages for the World Cup.

To make all this possible there has been fantastic support from the Sport team, engaging with new tools and workflows. We are all looking forward to the London Olympics, where there will be over 12,000 athletes and index pages to manage and so without this type of technology, we will not be able to showcase and maximise all the content we have.

A call to action

We'd like to engage further in the development of Linked Data and feel we have a role to play in supporting this important new view of how content is published and shared. The methods talked about here will become the basis for more and more of our content publishing and we fully appreciate the work many people are doing in this area to make this possible.

There is a vision for the future here with more time spent on creating and sharing content and less on managing it. However we have had to overcome many problems in getting this far and many of these issues are related to organising and cleaning up data. Due to all the technical and data challenges we have not yet been able to expose all our data as RDF, for example, though we will start doing this soon.

As more content has Linked Data principles applied to it (as outlined here , then these problems will become less significant and the vision of a Semantic Web moves closer. Importantly, what we have been able to show with the World Cup, is that the technology behind this is ready to deliver large scale products.

This is more than just a technical exercise - we have delivered real benefits back to the business as well as establishing a future model for more dynamic publishing which we think will allow us to make best use of our content and also use Linked Data to more accurately share this content and link out to other sites and content, a key goal for the BBC.

We look forward to seeing the use of Linked Data grow as we move towards a more Semantic Web.

John O'Donovan is Chief Technical Architect, Journalism and Knowledge, BBC Future Media & Technology. Read the follow up post, BBC World Cup 2010 dynamic semantic publishing on the Internet blog.

The BBC's approach to combating online piracy

Post categories:

Najma Rajah | 13:59 UK time, Tuesday, 6 July 2010

The passage of the Digital Economy Act (DEA) through the Houses of Parliament earlier this year highlighted the potential threat of online piracy for the creative industries. The legislation primarily focused on one approach to preventing online piracy: legal action against those who illicitly use peer-to-peer file technology to exchange unlawful copies of copyright material.

However, as has been apparent from the conversations on the BBC Internet Blog and elsewhere, other solutions also have a role to play such as the use of content protection technologies, development of new business models and reforming copyright law so that it is simpler to make legal alternatives to pirated content more widely available in easier to consume and more convenient ways.

So I thought it would be useful to take a step back and bring all these different strands of the debate together by summarising the BBC's broad approach to combating piracy in the digital age.

Piracy can manifest itself in many forms. For example, in addition to illicit use of file sharing technologies, streaming of whole channels over the internet and services which allow unauthorised access to BBC iPlayer by internet users outside of the UK can also pose challenges. It is important that the BBC takes a consistent approach to tackling all forms of online piracy. If we didn't, there would be a risk that dealing with one type of piracy might simply encourage interest in other types.

The principle of universal access lies at the heart of our overall approach to halting online piracy. A key objective for the BBC is to ensure that its content is available on as many platforms as possible for the benefit of licence fee payers. This suggests that the single most effective thing that the BBC can do to address piracy is to provide attractive legal alternatives - for example, by offering live TV and radio streams or by enabling licence fee payers to watch licence fee funded content on BBC iPlayer.

But in order to do this we need to need to balance many factors including value for money, market impact and our commitments to rights holders and the third parties involved in making our programmes, not only BBC Worldwide (as has been suggested on the BBC internet blog), but also actors, writers, photographers, musicians, sports rights holders, our partners from the independent sector and foreign studios.

The second element of the anti piracy jigsaw is promotion of copyright awareness and education. Over the longer term, online piracy can most effectively be tackled by changing viewers' attitudes and behaviour. This won't be easy as at the moment very little is known about who the most prolific online pirates are or what motivates them. Furthermore although it makes sense for rights holders to collectively undertake information campaigns about the impact of piracy on the creative industries, it can be difficult for rights holders who often have slightly different problems and priorities to agree coordinated action. This is something that has been recognised by government and in the future Ofcom will be required under the DEA, on an annual basis, to provide the Secretary of State with a description of the steps taken by qualifying copyright owners to inform, and change the attitude of, members of the public in relation to the infringement of copyright.

The third element is enforcement of anti piracy legislation. The BBC in the past has pursued businesses offering live or near live online streaming of BBC channels - for example TV CatchUp and Zattoo. However, for the first time, the Digital Economy Act provides opportunities for rights holders to pursue persistent offenders. As we have always made clear it is important that any legal solutions are both targeted and proportionate.

The final element of the BBC's wider strategy against piracy in the digital age, and the aspect that has prompted discussion on the pages of the BBC Internet Blog, has been the BBC's use of technical content protection. Earlier blogs have spelt out the rationale for activity in this area, so I won't repeat the arguments here. However, it is worth adding that since we last posted on this subject, Ofcom has concluded that we have struck the right balance between the interests of consumers and the interests of broadcast rights holders.

Looking ahead, the discussions are unlikely to abate for the foreseeable future. The implementation of the DEA will undoubtedly continue to provoke extreme views on both sides about how best to deal with the problem of online piracy. However, I hope as a result of this blog, the way in which any interventions by the BBC contribute to addressing piracy will be clearer.

Najma Rajah is Senior Economic Adviser in Policy and Strategy.

BBC News website redesign (1)

Post categories:

Steve Herrmann Steve Herrmann | 13:25 UK time, Tuesday, 6 July 2010

In the next week or so, we'll be making some improvements to the design and layout of the BBC News website.

Since our launch in 1997, we've worked to make sure the site continues to develop to meet your needs and expectations. This is the latest stage in that process of evolution.

We have focused on design and navigation, looking to see how we can make all the existing content we produce each day easier for you to find, use and share. I'd like to use this post to offer you a first glimpse; you can see a slideshow here.

Slideshow of the redesigned BBC News website

Read the rest of BBC News website redesign (1) on The Editors.

Round up: Tuesday 6 July 2010

Post categories:

Nick Reynolds Nick Reynolds | 11:30 UK time, Tuesday, 6 July 2010

When David Madden blogged that the iPlayer was now available on certain Google Android devices in June there was a discussion in comments which referred to (among other things) an unsupported third party app called beebPlayer.

Dave Johnston who made beebPlayer has now clarified a few points, in a post on his own blog: "Beebplayer's secret sauce". Here's some extracts:

The new iPlayer site does a much better job at providing a good experience for Android users than beebPlayer ever could have - not only are the streams it uses designed specifically for Android phones, but because of this they're higher quality and less likely to break at the merest sign of a lost bit or byte... The fact the new site is limited to Flash-enabled devices is a sore-spot, but I can understand this requirement and its benefits, as well as its current and future limitations. RTSP (mobile/beebPlayer) isn't exactly an ideal alternative if you have a level of service you wish to maintain, something RTMP (Flash) is far better at providing, especially across multiple different platforms... Despite what many people assumed, beebPlayer did not use the higher-quality iPhone MP4 streams - which are simply not quite compatible with Android (but only by a whisker - while Android phones are easily capable of playing the streams, they just don't quite understand the QuickTime container used.)
*

Last week paidContent revealed via a Freedom of Information request; "BBC iPlayer Costs At Least £5.2 Million A Year"

*

Meanwhile Anthony Rose, the man behind BBC iPlayer has been out and about talking about his new role at Project Canvas:

When asked what the priorities for Canvas were now that it had been given the go ahead by the BBC Trust, Rose said: "The priority for Canvas is to start building the sucker."
*

paidContent has read the BBC's Annual Report: "BBC Websites Cost Users £0.67 Per Month".

*

And Tom Service at the Guardian is very pleased that the BBC Proms Archive has now gone live in Beta:

Years of work by oxygen-starved BBC minions working in digital bunkers have resulted in the details of every single Prom ever performed - all 7,168 of them since Henry Wood first brought his baton down on a promenade concert in 1895 - being available now for your anorak-clad pleasure. At the click of a mouse, you can instantly discover the dates, times, soloists, conductors of the seven performances so far of Mahler's Eighth Symphony...

round_up_06_07-10.jpg

Nick Reynolds (pictured above in his 'digital bunker') is Social Media Executive, BBC Online.

BBC Trust's interim report on the BBC's Strategy Review

Post categories:

Paul Murphy Paul Murphy | 12:44 UK time, Monday, 5 July 2010

Today the BBC Trust published their interim report on the BBC's Strategy Review. Here are selected extracts about what the BBC does online:

"...a future where the BBC will have two distinct online functions - iPlayer will have an important role providing on-demand radio and TV-like content while BBC Online should also remain a publisher of distinctive, original public service material for the web. We agree the first step should be to simplify the existing content on BBC Online and to strengthen editorial control of its future development."

The 25% cuts in BBC online's budget looks certain to happen:


"We conclude now, in line with the BBC Executive's strategy proposals, that there is a bigger challenge for Online - to cut back on the current scale and scope of what is published and put the focus on those areas where the BBC has a clear and distinctive role to play."

But the report says more detail on how and where is needed.

The Trust stresses the importance of access to content for users:

"Get BBC services free at the point of use, in ways and on devices that suit them"

Simultaneously the BBC published its annual report.

Paul Murphy is the Editor of the BBC Internet blog.

Scaling the BBC iPlayer to handle demand

Post categories:

Simon Frost | 12:30 UK time, Friday, 2 July 2010

One of the key goals we set ourselves when we developed the new iPlayer was that it would have to be fast to use. We understand that any delay in getting you to the video is frustrating as the site is just a jumping off point into TV and Radio content.

But how do we make things fast? Displaying a web page in the browser contains many steps, some we can control some we can't. Time spent for the request and response travelling over the network we can't control, but we can control how long the pages take to generate and how large they are. We also have a degree of control over how long those pages can take to render in your browser.

We had our work cut out for us on the new version of iPlayer.

Personalised websites require much more processing power and data storage

The current site uses one back-end service that we pull data from to build the pages. The new site uses many more, and we both post and pull data from them.

This means that every returning user gets a different homepage. There's already a small amount of difference between each homepage on our current site (your recently played) but the new site is driven much more by your favourites, recommendations and friends; they're key parts of the experience and they have to be fast.

We started developing in PHP

The BBC is standardising on PHP as its web tier development tool. Our current site is developed using Perl and Server Side Includes, and it's something that's well understood, but our new web tier framework (based on Zend) means that teams can share components and modules. In fact, the team responsible for the social networking functionality develop modules that anyone within the BBC can integrate into their site easily.

This does come at a cost though: the usage of a framework sometimes introduces delay in generating a page as it needs to get hold of resources to do so. In some cases this is necessary, especially if there's an element of personalisation, but in others our web tier is just repeating the same tasks.

All this against a growing demand

The site will have to support a massive amount of page views and users every day, on average 8 million a day for 1.3 million users. Previous versions of the site were able to grow into this demand; we'll have to hit the ground running from day one.

page-views_595.png

This graph shows our growth over the last year in terms of monthly page views.


So how do we do this?

One of the first things we can do is optimise the time it takes to generate the page.

Although changing architectures can be risky, we were confident that the one we moved to would enable us to meet all the challenges. At the heart of page generation is a PHP and customised Zend-based layer called PAL. This system then needs to integrate with our login system, BBC iD, our programme metadata system (Dynamite), our social networking systems, a Key Value data store and a few others. The homepage alone for a logged-in user with friends requires 15 calls across these services. Even if each of those calls take a few milliseconds, we can spend a second or two just collecting the information required, which would push us well out of our 2.5s target.

We proved our architecture before we built it

At the start of re-architecting iPlayer, we did what we could to eliminate guesswork. We developed a number of architectures based on our requirements, and then built prototypes of three of them; all built to serve the homepage, which we then tested against some basic volumetrics. This gave us plenty of data about how many requests we could serve a second and CPU loads, which we could then weigh up against other softer factors, like how our dev team could work with it.

We actually ended up going for the one which offered us a good balance between these factors, as this enabled us to be the most flexible in building pages, rather than constraining what we could with the site just to squeeze the extra speed out.

We cache a lot

Caching means storing a copy of the data in memory so subsequent requests for that data don't have to do the expensive things such as database queries.

It also allows us to get around any delays introduced by our framework starting up, as there's no such delay when delivering from cache.

Caching has its problems though. The data may have changed in the underlying system (programmes become available to play for example) but the change won't be reflected in our cache. This means we can only cache for seconds or minutes, but with the millions of page views we get, it can still make a crucial difference.

  • Data caching We cache the data returned from the services. We use Memcached for this. Sometimes we share data between pages.
  • HTML caching We also cache the resulting HTML for a short time. When you're hitting a page, it's highly likely you're just seeing the cached page. We use Varnish for this. Caching in this way is nothing new, but Varnish has a few tricks up its sleeve that we use which I'll explain later.

We broke the page into personalised and standard components

If you look at our homepage, many of those components are the same for everyone, but some are just for you. With traditional page caching in some reverse HTML caches, it's not possible to do this; so we break the page up. The main build of the page is cached; then when the page loads we use XHR and Ajax to load in the personalised components. Varnish gives us the ability to control the caching at a low-level like this. Every time we generate a page or a fragment, we can tell Varnish how long we want to cache it for. The main bulk of the homepage doesn't need caching for long to get some benefit, but your favourites we can cache for longer (although still only for a few minutes), and we know when you add a new favourite so we can clear out the cache and replace it with the new content. This means as you browse the site, the page loads quicker and your experience is smoother.

We use loads of servers

After we've optimised all we can using a single server, we then scale horizontally using multiple servers joined together in a pool. None of our web servers store any state about who you are and what you're doing, so your request can go to any server at any time.

We also serve pages out of two locations (or data centres). This gives us a higher degree of resilience to failures; we can lose an entire data centre and still be able serve the site.

We load tested the site before we launched

We're able to track how the site is used, so this gives us the ability to produce detailed volumetrics of how we think the new site is going to be used. Some of it is estimation, but it's always backed up with data. We can then produce detailed load tests, so we can simulate usage of the site. This enables us to find and resolve any problems we may experience under load, before we go live.

The end result

We're not 100% there yet (this is a beta after all) but from this sample 24 hours of monitoring data you can see that, apart from a couple of spikes, we're doing well at keeping to our target of 2.5 seconds. (We were also able to track down the spikes to some misbehaving components on the platform).

page-times_595.png

We're currently working hard behind the scenes at making sure we can continue to serve at this speed as usage increases, spreading the load across our infrastructure.

At the end of this though, we hope the result of our efforts is that you won't notice a thing: it'll just work.

Simon Frost is Technical Architect for BBC iPlayer .

More from this blog...

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.