Future Media Standards & Guidelines

Development, Testing, Installation and Maintenance v1.23

This document details the development, testing, install and maintenance cycle for www.bbc.co.uk servers. For information on this cycle for news.bbc.co.uk servers, please contact the Product Lead, BBC News.

Top of page

1. Standard infrastructure and non-standard systems

The current bbc.co.uk infrastructure is documented in bbc.co.uk standard infrastructure – supported software.

If your project has requirements that do not fit in with the existing bbc.co.uk infrastructure you will need to discuss your options with your Technical Project Manager (technical lead). These may include:

  • Hosting by third party with links from/to bbc.co.uk services (see Third Party Hosting Requirements)
  • Installation of custom hardware to support your application within the bbc.co.uk architecture and with access to bbc.co.uk systems

In both cases security rules will apply and external independent 'Penetration Testing' companies will be used to check these systems in addition to the security processes below. The third-party provider will need to provide satisfactory evidence of good security/ policies and availability of their services.

For bbc.co.uk hosted solutions, all the rules in terms of access will apply. For externally hosted systems there will be more flexibility – if good practice is demonstrated.

Top of page

2. Development environment

2.1 Development servers

External developers may use the extdev.bbc.co.uk development server, which resides outside the firewall, for all testing purposes.

In circumstances where external developers are allowed to use development servers inside the firewall, you may use dev0.reith.bbc.co.uk.

2.2 Use of development servers

The extdev.bbc.co.uk development server is managed by the Siemens Internet Operations team. This externally accessible development environment is available for building and testing content and applications that will eventually be published as part of bbc.co.uk.

extdev.bbc.co.uk should not be used as a general storage area for material that will not be published onto the live environment. The server is not regularly backed-up, so developers are strongly advised to keep a back-up of all material published there.

A variety of features and options are available within the development environment, including:

  • www (for web pages, including Barley)
  • data (for templates used by CGI scripts)
  • cgi-bin (for CGI scripts)
  • results (only needed for CGI scripts that write files to the server)
  • cgi-perl (please see mod_perl guidelines in the (Server-Side) Application Development Standards Restricted access - Post-NDA only)

2.3 Development accounts

To apply for a development account on a development server, your Technical Project Manager (technical lead) must complete a Dev Account Request Form available from Siemens Internet Operations or Red Bee Media webmasters.

An account can take up to 3 days to process and set up.

When requesting a development account, consider the following points:

  • Will other developers need to work on content held within the developer's personal account?
    • If Yes: It may be appropriate to set up a generic "User Group" to enable read/write privileges to other named developers.
    • More information and docs on the environment can be found at http://extdev.bbc.co.uk/index.shtml

2.4 Development constraints

Ensure that you adhere to all the appropriate BBC New Media standards. Your Technical Project Manager or Technical Forum representative can assist you in identifying standards relevant to your project. These can be found at http://www.bbc.co.uk/guidelines/futuremedia/.

Ensure any database usage and schemas are approved by New Media DBA – as per the application standards

Inform the Technical Project Manager about volumetrics of your application – as per the application standards.

Ensure that your application works with the supported software versions installed on the BBC servers, or that you make a request to use unsupported software early in your development cycle – see bbc.co.uk standard infrastructure – supported software.

Ensure that file-paths will work after your project is moved to a BBC server. Typically these should be:

  • /home/system/www/[sitename]/ (for HTML pages)
  • /home/system/cgi-bin/[sitename]/ (for mod_perl applications)
  • /home/system/perldata/[sitename]/ (for templates read by modperl applications)

You MUST test your project on a BBC development server (e.g extdev.bbc.co.uk or dev0.reith.bbc.co.uk) before submitting it for acceptance testing.

2.5 Presentation of development content

When you set up an account, you can request the setup of a symbolic link between a developer's personal development space and that of the site under development. This will allow content to behave as if it were already on the live environment.

For example, if Joe Bloggs (with login username: joeb) is working on the "disco" site, Red Bee Media webmasters can arrange for the site to be browsable via the link http://extdev.bbc.co.uk/disco instead of (for example) http://extdev.bbc.co.uk/joeb/disco.

Top of page

3. Acceptance testing

3.1 Acceptance testing for new websites

All new bbc.co.uk websites MUST be tested against the appropriate bbc.co.uk technical and design standards before being published on the public web server, according to the acceptance testing procedure for website deliverables, as specified in the website's contract (see Legal & Business Considerations for further information).

Acceptance testing is supervised by Red Bee Media webmasters, who are responsible for monitoring and maintaining quality assurance by checking the consistency, continuity, and coherence of supported services. The team is responsible for testing the usability, accessibility, and cross-platform compatibility of supported services.

Sites that do not meet bbc.co.uk standards cannot be published. Website tests produce a report, available to the site's producer and developers, which outlines the reasons for a site's success or failure to meet these standards.

3.2 Acceptance (security) testing for new applications

To prevent downtime due to compromised applications, all active code destined for bbc.co.uk is vetted by the Siemens Internet Operations team to ensure it has no security flaws and passes the relevant guidelines [see (Server-Side) Application Development Standards Restricted access - Post-NDA only)].

Please liaise with your Technical Project Manager to assist in the operation of this procedure, when it is time to install an application to ensure correct process.

Executable code hosted elsewhere will be subjected to independent security testing prior to public release (see Third Party Hosting Standards).

Top of page

4. Uploading content and installing applications

To prevent illicit changes to bbc.co.uk site content or unauthorised use of personal data on bbc.co.uk, third parties only have access to the following:

  • An FTP account on the main distribution server to upload web, streamed and downloaded content and application configuration files (not applications)
  • HTTP/HTTPS to site and services with basic authentication where appropriate

Third parties have no direct shell access at all (e.g. SSH, Telnet or SQL*Net) to live applications, web or database servers.

4.1 Application installation

All applications are installed by Siemens Internet Operations – see (Server-Side) Application Development Standards Restricted access - Post-NDA only.

4.2 Uploading content via the FTPBorg queue

All new and updated content to the live site e.g. web pages, AV media and config files, are FTP'ed to borg.bbc.co.uk. The BBC uses a system called FTPBorg to ensure content is distributed to the correct set of live servers.

The directory structure in your FTP account determines the servers the content goes to. See 4.2.4 for further information on this. For example, content place in the 'rmhttp' directory of your FTP account, goes to the downloads servers 'downloads.bbc.co.uk'.

4.2.1 Gaining access to the MCS (FTPBorg)

If it is agreed that you can publish your own content and scripts, you MAY request a Trusted User Account from Red Bee Media webmasters. This account lets you upload content via FTP to the Master Control Server, giving you access to the content on the site that is suitable to the status of your trusted user account.

4.2.2 FTPBorg queue upload etiquette

The FTPBorg queue is a facility which is shared by the webmasters of all websites to update their content. To ensure that no individual webmaster abuses this contended facility, you must follow these guidelines whenever you upload anything to the queue:

  • always upload the minimum number of files necessary to make your content change – for example, do not upload your whole site to make a small change to a single page.
  • Before uploading a large volume of files during a peak time, you must contact the Red Bee Media webmasters to request their permission.

4.2.3 Upload process

  • FTP the files to the correct directory on borg.bbc.co.uk. (NB. Always save text filetypes as UNIX (not Windows) and always upload text file-types as ASCII).
  • Check that they appear at http://www0.bbc.co.uk.
  • The Borg system intercepts your FTP requests (such as PUT, DEL, and MKDIR), then puts them into a transaction log file, which is applied to all servers in the content network in the order received. Find out how long your content update will take to reach all live servers by viewing http://www0.bbc.co.uk/cgi-bin/support/borgqueue. Please ensure your upload application is resilient and restarts in event of a borg restart/recovery.
  • Examine your changed files at http://www.bbc.co.uk.

4.2.4 Architecture of FTPBorg

The architecture of the FTPBorg system is straightforward. FTPBorg places new content in a queue for upload to each live server. Upload takes place in parallel to each server, but (depending upon a number of factors) can cause content to be delivered over approximately 15 minutes. This is especially the case for our international servers.

Not all files placed on borg.bbc.co.uk are borged to all servers. There are five distinct server types in the network.

  1. Web content visible to the public from www.bbc.co.uk.
  2. Modperl applications running on dedicated application servers, visible via www.bbc.co.uk/cgi-perl/.
  3. CPSs* dedicated servers to generate content for the main site.
  4. Audio or video Files for streaming servers (Windows and Real format
  5. Large file servers ( 500kbytes+ ) (downloads.bbc.co.uk)

Your FTP account displays the following directories and their locations.

  • /www/ – 1 – URL: http://www.bbc.co.uk
  • /data/ – 1,2 – URL: Not visible
  • /perldata/ – 2 – URL: Not visible
  • /cps/ – 3 – URL: http://cps0.mh.bbc.co.uk/...
  • /results/ – this is a special directory showing the output of scripts needing to write to files – URL: Not visible
  • /cgi-bin/ – 1 – URL: http://www.bbc.co.uk/cgi-bin/...
  • /cgi-perl/ – 2 – URL: http://www.bbc.co.uk/cgi-perl
  • /advance/ – in this folder, you can create a sub-folder where the name is the time and date at which you wish that content to become active. If you would like to use this feature (which is good for launching content to synchronise with timed events like programme broadcasts) then contact Red Bee Media webmasters to set up the advance folder in your trusted user FTP account. – URL: Not visible
  • /realmedia/ – 4 – see AV standards
  • /rmhttp/ – 5 – URL: http://downloads.bbc.co.uk/...

It is important that you place the correct content in the correct folder above. For example, streamed AV content will not work unless placed into the 'realmedia' folder and large file content (500Kbytes+) must be placed into the 'rmhttp' folder.

If any of the above folders do not exist in your account, request Red Bee to set them up for you.

Top of page

5. Support and maintenance

When planning your project you MUST consider issues of support and maintenance and document and supply the relevant information to the technical contact, such as:

  • What will need to be maintained?
  • How much will need to be maintained?
  • Who will maintain it?

For applications running on http://www.bbc.co.uk, or for developers producing content for it, the following support and maintenance arrangements are available:

  • First line – Red Bee Media webmasters
  • Second Line – Siemens Internet Operations, who offer the following services:
    • Various RRD based monitoring and reporting tools
    • A hotline number for 24/7 technical staff with direct access to these systems (the hotline can be used to request application changes or installation)
  • Third Line – the following groups support their own applications:
    • New Media developers within BBC divisions
    • External companies, where appropriate

If you are developing applications for www.bbc.co.uk, then you MUST plan for first, second and third line support. You MUST provide those teams with documentation and possibly training so that they can carry out typical support activities. If you do not intend to provide third line (maintenance) support or ongoing development of the application, then you MUST have an agreement in place with the team that you expect to provide this for you.

Top of page

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.