How we built the new BBC Homepage

Senior Web Developer, Content Discovery

Tagged with:

The new BBC mobile Homepage is a flexible ‘taster menu’ of the BBC's website. I’ve been leading the development of the new Homepage and in this post I explain some of the technical design decisions made during the initial phase of the project. Please note that some of the language in this post is intended for readers with an interest in software delivery.

The new homepage released last week is the first version of a product that will be continuously improving. Many advanced features are still being developed, such as user customisation; these will be progressively integrated into the application through a process known as Continuous Delivery.

Customisation in particular is one future feature that drove the technology decisions made early on in the project. The new Homepage will allow our audience to change the content shown to them based on their preferences and location. To achieve this we require a flexible product that can handle high load with a large number of variants of the page - a different variant for every user's preferred configuration.

A new platform

We chose to move away from the Forge platform (a web application development platform that is hosted on physical servers using a recommended set of technologies such as PHP and Java) to a cloud infrastructure. This allows us to respond to demand by adding servers and save costs in quiet times by removing servers.

This new platform also allows us to add features much more quickly. This is partly because our developers now control the deployment process rather than a centralised operations team. Instead of releasing production code every two weeks, we are now able to release many times a week. As our process improves we expect to be able to speed up our release cycle even further.

This continuous delivery program empowers developers to act in a 'DevOps' capacity - as well as building the application, we also take charge of deploying, maintaining and supporting the infrastructure, in and out of office hours.

A new language and framework

Another benefit of the new platform is that it makes it easier to use languages and frameworks that best suit the needs of the project. Rather than using PHP or Java (that was the requisite of the Forge platform), we have chosen a non-blocking framework, NodeJS with the Express framework  . This allows us to serve more simultaneous requests, increasing the performance of the application. As a result, we can serve multiple variants of the page without needing to rely on HTTP caches such as Varnish; we cache each module on the page in Redis and construct the page using the cached fragments instead.

NodeJS was chosen from a list of candidates that included Scala and Clojure, as it performed well under load tests during a prototyping phase. Another benefit is that Javascript is well known to Web developers, and because both our server side and client side now utilise it there is less need to context switch when developing the product.

Taking a data driven approach

The Homepage is designed to surface content from all over the BBC – we do this in two ways. The large image-led promos at the top of the page are created by our editorial teams in a traditional CMS called iSite 2.

An example of an editorial created promo

The headlines in ‘Latest News’ and the clips in ‘TV’s most popular’ are automated feeds derived from many APIs across the BBC. The wider Homepage team has created a service called Curation Kit that allows our editorial teams to select and mix feeds from systems such as CPS (our journalist CMS) and Nitro (our programmes information system). The service combines them together into ‘Curations’ which can then be consumed by applications such as the new Homepage.

Curation Kit is built using  Scala on the Play framework , with a web interface built in  AngularJS . The service itself is made up of several micro-services deployed on a cloud infrastructure, backed by a MySQL database. Faith Mowbray will be telling you more about Curation Kit in a forthcoming blog post.

The Curation Kit user interface

Increased modularisation

Our editorial teams can now choose from a wide selection of modules to add content. Editors toggle whether stories should come from Curation Kit feeds, or from our CMS system, iSite 2, for editorially produced promotions. In the future an increased number of available modules, feeds and promos will allow us to deliver customised content based on your preferences.

The module content form on our internal CMS, iSite2

By separating these modules into micro-services, we will be able to reuse them across different applications and pages in the near future.

New front-end technologies

On the front end we have embraced new technologies such as isomorphic Javascript using ReactJS. This allows us to execute our Javascript templates on both the server and the client meaning that users without Javascript can still load a basic version of the page. The technology will also allow us to quickly develop modules that can periodically update themselves with live data from around the BBC. All our scripts are bundled together using Browserify, which allows us to mix both the  AMD and the CommonJS module loading systems. Utilising a mix of these two loading systems means we are not restricted in our choice of open source libraries because of the mechanism implemented in a particular module.

We are using Sass  to build our stylesheets and BEM  to structure our CSS selectors to ensure good performance. The Sass stylesheets are compiled to CSS during our automated build process, which runs using Gulp , our build system, both locally and on a Jenkins continuous integration server.

Gulp also runs our automated unit and acceptance tests, which are written using Mocha, and various other tasks such as linting and static asset compilation. This level of automation allows the developers to quickly verify that new features going into the live product have not caused problems with the rest of the application. This ensures the user always sees the page as intended.

A snippet of the output from our automated tests

We are building the Homepage mobile-first, using responsive design with responsive images to ensure an optimal experience and good performance on mobile devices, tablets and soon, the desktop. Keep an eye on the page over the coming months for the desktop version, and new ways to find and view the range of stories the BBC makes online.

If you'd  like more information about any of the above please do leave a comment.

And if you're a software developer and are interested in working for the BBC online you can search for web developer and software engineer roles on the BBC careers portal .

Tagged with:


More Posts