Server-Side Application Development Standard v2.0

1. Introduction

This standard is designed to ensure that all server-side application development maintains the integrity and security of the BBC website. This standard is applicable to development on ‘The Platform’ (previously known internally as Forge). It is not applicable to development on the Siemens and News servers.

This standard links out to information available on the BBC’s Confluence wiki. External developers can gain access to this restricted documentation via their digital certificate. See the BBC's, Welcome to the Dynamic Web Application Development Platform pages for more information. Restricted standards and guidelines access is available via the Editor, Standards and Guidelines.

Top of page

2. Design and Planning

2.1 General

As early as possible in your project (before starting to code) you MUST consider the following issues.

2.1.1 You MUST confirm that the functionality does not already exist. Make sure you ask the right questions at the technical kick off meeting.

2.1.2 You SHOULD consider whether a client side (Flash or JavaScript) solution would be more appropriate.

2.1.3 You SHOULD use Java for the service layer and you SHOULD use PHP for page assembly layer. These MUST be used within a well-defined architecture. The BBC has adopted an SOA approach and URL namespaces SHOULD be RESTful.

2.1.4 You MUST ensure your storage solution is appropriate to your application, e.g. flat files, key value store, databases, etc.

2.2 Resource Planning

2.2.1 You MUST scope out the patterns of usage for your application. Will there be particular a time when usage will be very high? Is it tied to a promotional campaign?

2.2.2 You MUST have a cacheing strategy. Please see the BBC wiki pages: Platform Caching Service and HTTP Caching.

2.2.3 You MUST use tools that are available, e.g. mod_cache, memcache, varnish, identity, the key value store - do not reinvent the wheel.

2.3 Connecting to other hosts

2.3.1 If you need to connect to another host you SHOULD treat it as carefully as a filesystem operation, and be aware of compromising the host you're connecting to, or holding connections open.

2.3.2 You SHOULD implement a list of allowed hosts, and check against this prior to connecting, or hardwire the hostname into the code.

2.3.3 You SHOULD consider issues like denial of service (including ones that could be caused by your application), and how your application behaves when trying to connect to very slow, or dead hosts.

2.4 Email

2.4.1 If you application needs to send email, then you MUST refer to the Email standard.

2.5 Sending system information to the user

2.5.1 You MUST NOT send system information, such as paths, to the user as this information is useful in trying to break the security of the server.

Top of page

3. Security

3.1 Your application MUST NOT do anything that may compromise the security of the server, data on the server, other applications, other hosts, or the network.

3.2 You SHOULD consider all data read by your application whether from the user, the file system, or a database to be untrustworthy. Don't assume data you read in is in the correct format, is of the correct length, or is correctly delimited or terminated, or is even there.

3.3 When accepting data from untrusted sources, such as user-generated content, you SHOULD validate this data as it may contain code which could then be executed at a later time (e.g. Javascript contained in a form submission).

3.4 For applications that process personal data you MUST adhere to the requirements of the Technical Implementation of DPA.

3.5 You MUST NOT modify environment variables in your code. Also be aware that some variables, for example HTTP_REFERER are easily spoofed by user-agents.

3.6 You MUST fully consider security for admin applications See the Baseline Security Standards.

3.7 Applications MUST ensure that all filesystem access is carried out securely (this applies to all filesystem operations, not just reading and writing, for example changing permissions, moving files, or creating directories).

3.8 You MUST protect your sensitive data while in transit and at rest, via encryption etc.

3.9 If you are handling data you MUST ensure that you validate your input (see 3.3)

3.10 You MUST NOT fork other processes. Use built in methods, or standard libraries instead of forking.

3.11 You MUST consider deadlock/DoS (denial of service) situations and you MUST programme defensively.

3.12 You MUST NOT evaluate code at run-time.

Top of page

4. Scalability

4.1 General

4.1.1 Specific resources MUST be accessed by service name, and MUST NOT be accessed by server name, e.g. www.bbc.co.uk instead of www.live.bbc.co.uk.

4.1.2 Stateful applications SHOULD use cacheing and avoid server hits where practical.

4.2 Load balancing

4.2.1 An application MUST be capable of being load balanced across multiple servers and across multiple data centres.

4.3 Efficiency and Reducing I/O (Input and output)

4.3.1 If you parse a file repeatedly you MUST cache the parsed data.

4.3.2 Reducing I/O (Input and output) is often a good way to make your application scale further.

4.3.3 You MUST ensure that HTTP cache headers are set appropriately for the response from your application (see W3C's Hypertext Transfer Protocol - HTTP/1.1 for the full explanation).

4.3.4 If you are calling an HTTP API endpoint then you MUST respect the cache headers in the response and use conditional requests if they are supported by the API.

4.3.5 Your application MUST be able to cope with unavailability of other services, rather than passing on all requests to a broken service because your cache is empty. If you are calling a data source that is unavailable or temporarily down then you SHOULD NOT repeatedly spam the data source.

4.3.6 Some data sources are replicated across multiple data centres so you SHOULD only write to replicated data sources if you actually need to, as by writing data you will be contributing to the general replication lag.

4.3.7 If you are storing a large amount of data in a single cache key then you SHOULD consider compressing the data.

4.3.8 If repeatedly calling the same file, memcached key or database query then you SHOULD consider caching the contents for a short while in local memory. If your application is doing 100 reads per second then even a 1 second cache will save a significant number of reads.

Top of page

5. Maintainability

5.1 Documentation

5.1.1 All code MUST be well-documented. See BBC wiki page, Documenting your application or product.

5.1.2 Use the inline language-specific comments where you can, e.g. phpdoc, javadoc.

5.2 Logging and error reporting

5.2.1 You MUST NOT expose personal or sensitive information (such as passwords) in logs or in an any data returned to the client.

5.2.2 You MUST NOT expose information useful to malicious users (such as filesystem paths, or network topology) in data returned to the client.

5.2.3 Messages returned to the client (whether a person or another machine) SHOULD contain the minimum amount of information necessary to satisfy the request. When returning a message to the client, only provide information that is relevant to the client's request and do not add extra detail or granularity that is not necessary. Log messages MAY contain relevant information appropriate for developing, debugging and reconstructing the cause of errors (but are still subject to to the requirements of 5.1.

5.2.4 Applications in live production SHOULD have quiet logs in normal operation. For more information please see wiki page, Environment Based Logging. Environments provide an indication of the expected log level threshold which SHOULD be respected in an application's logging configuration. Development environments have a lower log threshold and are expected to have relatively more log activity, e.g. debug or info messages that are useful to developers and testers.

5.2.5 You MUST use the standard method for logging on the Platform according to programming language:

• use log4j in Java, appending to standard output;
• use Zend_Log or error_log() in PHP.

See wiki page, PAL Logging FAQ for more information.

5.2.6 You MUST use the appropriate log level for each message - e.g. in Java log4j uses the levels 'debug', 'info', 'warn', 'error'.

5.2.7 Error logs MUST only contain relevant error messages and you MUST NOT spam error or application logs with debugging information or excessive warnings. Consider that Operations Engineers will have to read your application logs.

5.2.8 Log messages SHOULD contain other relevant information such as an unambiguous precise timestamp, an indication of the class and method name, and the log level. For further information, please see wiki pages The Sample Service and Logging Best Practice.

5.3 Clean-up

5.3.1 Applications that write any data to disk, especially for caching, MUST ensure they clean up after themselves. If a separate cleaning script/cron is used, it MUST be installed before the main application is used.

5.3.2 When using caches, you SHOULD implement a way to clear or disable the cache that you can control.

5.3.3 Your application SHOULD only write to temp files if it needs to and your application SHOULD handle write errors, such as if /tmp has filled up.

5.3.4 Your application SHOULD tidy up after itself and remove temporary files that are no longer needed so that the partition does not fill up and prevent your and other applications from writing.

5.4 Provision of automated monitoring

5.4.1 You MUST ensure a suitable level of automated monitoring is in place for your application. See wiki pages Platform Monitoring Service and PAL Monitoring for more detail. See also wiki page Monitoring Application Internally which gives specific information on developing monitoring alerts on the platform for your application.

Document history

Top of page