Programme Styles

The last area to talk about is programme style and the techniques used by producers. The HD Channel is expanding the range of programmes made and transmitted in high definition all the time. As the number of programmes grows we are bound to show some that don't appeal to everyone in the audience, just as any multi-genre channel would. Similarly some of the techniques and styles used will not match some viewers' expectations of what is or is not HD.

Posts have suggested we are changing the bit rate depending on the programme because individuals have seen something they really like the look of while others have suggested programmes are up-converted when they don't like the look. Neither is true. The bit rate has been constant since the change in August and the amount of up-conversion (or non-HD) allowed is still 25%. There is though a correlation between comments posted and some of the production techniques used to make HD programmes.
Techniques that cause the most debate are:

• Motion portrayal (using 25p or so called "Film Motion")
• Depth of field or focus
• Noise & Grain
• Softness & Smear

Motion Portrayal - Many programmes are shot using a camera's 25p option i.e. film motion. Drama and Natural History moved to HD from film and wanted to keep the same look. Technically there is more detail in a progressive image than the equivalent interlace image. I have mentioned this before in several posts.

Temporal resolution describes the motion of a sequence i.e. the duration of each frame in a moving sequence and therefore how far an object moves between frames.

Spatial resolution describes the detail of the image and is about the content of a frame.
The BBC R&D white paper WHP169: High Frame-Rate Television gives a very interesting insight on the relationship between the two.

At the moment though we leave the decision about motion type to the Producer and Director of Photography of the programme but there are times when we do comment on the inappropriate choice of 25i or 25p.

Caption: These two images are from the white paper. They are frame grabs of a moving train shot at 300fps (bottom image) and translated to 50fps (top image).

It is worth reading the paper if you want to know how the detail you can see in an image is very much dependant on the frame rate - unless there is no movement. However we are not talking 25 to 50fps to make the difference in clarity you see here it's more like 25 to 300fps!

Depth of Field - Drama productions often use focus to move the point of interest in a frame. Programmes such as Cranford and Wallander also use feature film style cameras with a very small or shallow depth of field which allows the point of focus to play a major role in the story.

There have been many comments about HD being pin sharp and some people believe an HD image should be in focus from the nose of the person in close up to the trees on the horizon. I think many people would find that incredibly distracting and (as some of the posts lead me to believe) people would just be looking at the image quality not the programme. Focus is a very useful programme-making tool, and when used well it adds to the look and feel of a programme.

• Depth of Field entry on Wikipedia
• Alan Roberts' book Circles of Confusion is a far more in-depth look at HD cameras and covers depth of field in Chapter 11.

Noise & Grain - as the range of programmes on the channel increases, we get programmes that are deliberately made to look "dirty" either by style of shoot or during post production. This technique is often used in the cinema too. However if overdone for television it will stretch the encoder and cause unpredictable quality changes. Programmes that do this will usually fail a technical review.

Softness & Smear - some programme makers do not want extremely sharp images and choose to soften the picture either with lens filters or in post production. Both series of Criminal Justice for example used softening to create a very distinctive look. However if this is overdone it will significantly reduce the image resolution and can increase the amount of noise in the image. We do try to limit the amount of lens filtering programmes use but if extreme image softening is required, we encourage people to it in post production so that if the end result is unsatisfactory at least we can ask for it to be removed.

Motion blur (or smearing) is another matter. It usually occurs when the camera shutter is not set appropriately. A programme shot at 25p should use a shutter speed around 1/50th sec (or 180˚). If the shutter is not turned on or is set too long, the images will smear. This doesn't look very nice and we do try and stop people doing it. If the producer wants to add motion blur, again it is always better to do it in post production. Doing it in camera is usually fatal!

Last thoughts:
It has been a marathon blog and has taken me a long time to write! But there was a lot to go through and a lot to check before I published. Over the last six days I have covered most of the technical issues raised in the four key picture quality blogs. I hope it has answered the questions you've asked and laid to rest some of the rumour.

Running through the sections day by day

I decided audio is always forgotten or left 'till last so I put it right at the front. I covered the recent audio issues we've had and some of the measures put in place to prevent them happening again if at all possible. More programmes are being delivered in surround sound now and I hope some of the recommendations due from the EBU next year will give people more confidence to try it out.

There is one thing that does crop up from time to time that I need you to watch (or should that be, listen) out for. Very occasionally the 5.1/2.0 switching gets stuck. It works very well for months then for no apparent reason sticks in 5.1. This has no effect on the audio but it's annoying for anyone using the switch information go to the Pro Logic option on an AV amplifier. Please keep letting me know if you spot it though and which programme is affected.

On Monday and Tuesday I looked at the history of HD encoding on the channel and the EBU recommendations covering programme making and transmission decision we made when the channel started. We review these decisions regularly and changes are made as and when the technology allows us to so.

Wednesday's and Thursday's posts covered how the tests are set up and the much requested PSNR and expert viewing tests results. It is fairly clear now that the new encoder is a lot more efficient than the old and more importantly, capable of many further upgrades as the technology continues to develop.

There is no doubt this series will cause much comment and raise more questions about image quality and encoding generally. I will try and answer as many as I can but in the end we may have to admit we will never be able to satisfy all of you when it comes to what is or isn't high definition.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

• Read part 1 of Andy Quested's HD guide: The Hitchhiker's Guide to Encoding: Before we start
• Read part 2: The Hitchhiker's Guide to Encoding: Life, Encoders and Everything (Or a brief history of HD encoding)
• Part 3: The Hitchhiker's Guide to Encoding: So Many Tests, and Thanks for All the Recommendations (Or the BBC and the EBU)
• Part 4: The Hitchhiker's Guide to Encoding: Mostly Testing (Or how to set up an encoder test)
• Part 5: The Hitchhiker's Guide to Encoding: And Another Test...(Or PSNR and all that...)

PSNR Measurements

This bit is all maths and I make no apologies for it! It is one of the methods used to evaluate the effect processing has on signals and ultimately picture quality. PSNR is a derivative of the Signal to Noise Ratio comparing the maximum possible signal energy to the noise energy ^[1].

PSNR has been shown to have a high correlation to subjective picture quality (eyeballs) when a single codec is used and cross-references between sequences are not made ^[2].

Equation 1- the Mean Square Error (MSE) of the mth frame is calculated ^[3]. Y_in and Y_out represent the luminance of the input signal from the play out server and output from the encoder respectively, and Y(I,j,m) is the luminance value of the pixel in position (I,j) in the mth frame.

Equation 2 - the PSNR of the mth frame is calculated ^[3]. B is the number of bits per sample used in representing the video. The test procedure uses 8-bit linear pulse code quantisation.

In accordance with industry recommendations, only the luminance PSNR is measured ^[3]. Typical values for the luminance PSNR for emission encoding are between 30 and 40dB.

This is capped to a maximum figure because an 8 bit system cannot accurately represent the original, analogue video image. In practice, the industry recommendation uses a cap of 50 dB ^[3], almost 10dB lower than the theoretical maximum. Above 50dB the quality of the coded image is more than sufficient for all but the most critical applications.

The median PSNR is the value of the 50th percentile of the individual frame PSNRs of a sequence listed in ascending order. The accepted critical value for this type of measurement (as used by the Moving Picture Experts Group (MPEG) in deciding if a toolset should be included in an implementation) is 0.5dB.

0.5dB represents a visible difference in picture quality across the range of PSNR values. As the PSNR increases and coding errors become less visible, the visibility threshold increases above 0.5dB.

• ^[1] L. Hanzo, P. Cherriman, and J. Streit, Wireless Video Communications - Second to Third Generation Systems and Beyond, ser. Digital and Mobile Communication. 3 Park Avenue, New York, NY, USA: The Institution of Electrical and Electronic Engineering Press, 2001.
• ^[2] Q. Huynh-Thu and M. Ghanbari, Scope of validity of PSNR in image/video quality assessment, IET Electronics Letters, vol. 44, no. 13, pp. 800-801, June 2008.
• ^[3] Objective perceptual multimedia video quality measurement in the presence of a full reference, International Telecommunications Union Telecommunication Standardization Sector - Pre-published Recommendation J.247, August 2008

PSNR of the current encoder setting compared to the old encoder.

The material used to test the encoders was a selection from the EBU test sequence and clips from the BBC HD Channel promotion.

All test material is copied to the playout server (100Mbs MPEG2 I-frame coding) and then onto the transmission encoder. The Final Cut Pro computer is used as a store for the transmission decoded material.

The results are displayed as curves on a cumulative graph. The x-axis is the measured PSNR and the Y-axis indicates the percentage of frames with a PSNR value less than or equal to that PSNR:

The new encoder has a median PSNR figure 0.5dB greater then the old, a just noticeably improvement in perceived picture quality for the majority of the test sequences. However the very easiest, least critical material, where coding artefacts are usually not visible, coded with a better PSNR on the Old Encoder. We are looking into this at the moment but one explanation could be the new encoder handles image noise differently to the old encoder.

Reading PSNR curves is not straight forward. A difference of about 0.3dB is just visible to an expert viewer at normal viewing distance while a non-expert viewer will see a difference of 0.5dB or more, as mentioned in the last paragraph of the PSNR explanation.

Where differences occur in the curve is important, at the lower end (the further left you go) the more critical the measurement. At normal viewing distance a non-expert may see a difference in quality for a change of 0.5dB or slightly less. At the far right of the curve the picture quality is much higher and differences are more difficult to see so an expert may not see a difference under 0.5dB and a non-expert may not see any difference below around 0.75dB or even 1dB.

Mix/Fade problem

We were aware of a problem with mixes before the new encoder went into action. During tests it only appeared in certain modes and wasn't severe. The overall improvement in quality outweighed the degradation it caused.

Unfortunately one of the first live programmes to be transmitted was also a programme that would highlight the mix/fade problem.

The Match of the Day, West Bromwich Albion vs. Newcastle United game kicked off with a very high contrast change almost dead centre of the pitch. As the game moved from bright sun to deep shadow the cameras had to be racked over several stops (opening and closing the iris).

A mix as you know is a transition between two different images. Coding errors caused by the mix tend to be hidden by the changing images however racking a camera is actually a mix between two different brightness levels of the same image so there's no where for the errors to hide and they become very visible. I apologised and explained we were applying a temporary fix.

Although the temporary fix is still in place we have now seen an update that improves mixes, fades and lighting changes and are just waiting for it to be incorporated into a software upgrade.

While we had the location recordings of the match to analyse the mix error, we had a chance to compare the PSNR curves through the new and old encoders:

For the majority of the sequence, the new encoder has a higher PSNR than the old with a median increase of about 0.2 dB (not a noticeable difference). The old encoder is better for approximately 8% of easy to encode scenes and 1% of difficult to encode scenes, but this is most likely due to the camera racking i.e. the mix/fade issue itself!

PSNR testing shows the new encoder is doing better than the old except where the source material has a significant amount of noise. To help this we are testing the encoder's noise reduction options to see if adding a small amount improves the look of noisy images. I will update the blog as soon as we have some results.

Subjective expert viewer evaluation

The second part of the testing process is all about looking at pictures. We use 42" plasma and LCD displays to do this, comparing the quality of the new and old encoders against the original material on the play-out server.

Expert viewing is a tricky business and as one of our experts discovered a risky one too! It involves watching the same set of images again and again and again and...

To minimise the risk of complete insanity, it is usually better not to have the audio on.

However even this didn't prevent someone coming in to work one day and asking to be taken off picture evaluation for a while. He said was on the train just looking at the country side passing by when he was convinced he saw compression blocking in the leaves of trees. This is not something you want to happen - so be warned!

Evaluating picture quality this way means a long time spent in darkened rooms. We watched a lot of images from the EBU test material and the BBC HD promo tape, comparing the new encoder with the old encoder and the play-out server on each sequence. It is important to have the EBU standard sequences to judge picture quality but we also have a test sequence made up from material that has known problems and shots that are difficult to code, to test the encoders to the "limit".

The new encoder produces images that correlate quite closely to the PSNR results. Programmes with low or no noise are noticeably better than they were on the old encoder. However, where the original images have noise we can see it on the new encoder's output but not on the old, suggesting that the new encoder is attempting to pass on more of the original image and confirming that a bit of noise reduction should be tested.

Dark pictures are inherently noisy, either because there is gain in the camera or the signal has been stretched too far in colour grading. We actually have a very noisy sequence that has too much camera gain and was stretched too far in post production. We used it during the tests to push the system, and even turning the bit rate up to just over 16Mbs made no difference to the image. We are trying a few new and different parameters that seem to improve noise handling and reduce the effect on screen. Again I will keep you posted.

Tomorrow is the last part of this epic, I will look at some of the techniques programme makers use that can have an impact on perceived picture quality.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

• Read part 1 of Andy Quested's HD guide: The Hitchhiker's Guide to Encoding: Before we start
• Read part 2: The Hitchhiker's Guide to Encoding: Life, Encoders and Everything (Or a brief history of HD encoding)
• Part 3: The Hitchhiker's Guide to Encoding: So Many Tests, and Thanks for All the Recommendations (Or the BBC and the EBU)
• Part 4: The Hitchhiker's Guide to Encoding: Mostly Testing (Or how to set up an encoder test)

Test results can only be seen to be accurate if as many variables as possible have been removed or at least minimised, and the methodology produces repeatable results.

Our picture quality assessment is based on a combination of Peak Signal to Noise Ratio (PSNR) measurement and expert viewing. Obviously, if done correctly PSNR measurements are accurate and repeatable within a very small tolerance range but the results are in the end only numbers. Eyeballs on the other hand, even expert ones, are slightly less predictable but can still produce repeatable results within an agreed tolerance but eyeballs are essential to judging overall picture quality of any device.

Removing Variables
Ideally the only variable in a test should be the device actually being tested. In a delivery chain with domestic devices from many manufacturers this is not as easy as it sounds and testing every combination of is just not possible.

To minimise the number of variables we use:

Test material - EBU test sequences used by all members for subjective picture quality testing.

Test Path - this is a duplicate of the actual transmission chain and includes the playout server and the continuity path. The playout server can also be bypassed to simulate live studio programmes.

Encoder - the bit we are testing!

Receivers - this is slightly more difficult because processing in set top boxes varies between different manufacturers and different models from the same manufacturer. For encoder comparison testing though we can use broadcast quality receivers. We also use domestic set top boxes from several different manufacturers to make sure the domestic receiver technology does not cause unexpected results.

Displays - this is even more difficult than dealing with receivers. We should use a graded broadcast monitor (a display where all the parameters are know and are adjustable to give a stable accurate image). There is still some debate around flat screen Grade 1 displays - until recently CRT was the only option and they're not easy to get hold of now!

• http://broadcastengineering.com/test_measurement/broadcast_monitors_video_801/
• http://broadcastengineering.com/news/replacing-crt-1005/index.html

In practice we have to think about what people are actually watching at home so we use 42" Plasma displays for comparisons and Plasma and LCD for quality assessment.

A 42" display is slightly bigger than the current highest selling TV size so this gives us a bit of head room during the assessments. The results will remain valid until the TV size passes 42" or the panel display technology changes dramatically (e.g. OLED or Laser etc.) and a new one takes over as the primary panel technology.

Viewing distance - The ITU has set out criteria for viewing distance in the document ITU-R BT.500 "Methodology for the subjective assessment of the quality of television pictures"
In the section "General viewing conditions for subjective assessments in home environment" it suggests preferred viewing distance (PVD) measured in screen heights (h) for assessment of picture quality.

The PVD suggested for 16:9 screens are:

The BBC HD Channel expert viewing is done at 4h (four times the height of the display used) so slightly closer than the ITU recommendations. This is based on the premise that the average domestic viewing distance is somewhere between 4 and 6h. Some interesting work done by BBC Research (WHP090) suggests the most common viewing distance is actually 2.7m no matter what the screen size. This is not a "serious" piece of work, more an observation of current viewing habits!

How to calculate your viewing distance?

There are two ways to calculate the height of a 16:9 television:

1. Televisions size always refers to the diagonal so first calculate the angle of the diagonal (Θ°):

Θ°= 29.36˚ All 16:9 televisions should have the same angle no matter what the screen size.

Using the angle calculate the height (h) of the television knowing the screen size:

so (h) is near enough half the (Screen Size). Multiply this by 4 to get the viewing distance.

Unfortunately this assumes all televisions are actually 16:9 which they are not - so just in case method 2 might be more accurate!

2. Use a tape measure
Or just use the table below!

When the 4 - 6h criterion was originally devised, screen sizes around 42" would have been considered extremely unusual. Now 42" screens are common and could soon be the highest selling set size. Due to the distance between the television and the back wall of the room you watch it in; WHP090 may prove to be spot on!

In the quality viewing area at BBC Research there is a mix of set top boxes and display types. Viewing distance is set at 4h.

Eye sight - before the BBC HD test channel started, a few of us spent several days in a very hot room above a shop by Oxford Circus doing viewer testing of HD and SD. We had two top end (at that time) 40" flat screens and a 28" CRT, the highest selling TV size at the time. We also had a selection of BBC HD test material as this was before the EBU had agreed test sequences.

The pictures were coded in HD using the encoder profile we proposed to go on air with and also in SD using the BBC 1 encoder and bit rate. The HD signal was fed to one LCD and the SD signal to the other LCD and the CRT.

After the first day we had a number of results that suggested no difference between the SD and HD LCD - quite confusing because even viewing at a considerable distance there was a marked difference between the two LCD images.

The next day we took a standard eye test chart and asked anyone who couldn't see a difference between the two LCD screens if they would mind taking an eye test. Everyone who took the test should not have been driving without glasses and many of them didn't event realise they should see an optician!

Anyway the expert viewers do have eyesight within the normal sight range (with glasses if they usually use them for watching TV).

This sets out the criteria for visual picture quality assessment but as I said earlier, testing uses a combination of visual assessment and a Peak Signal to Noise Ratio (PSNR) measurement.

Tomorrow will be all about the picture quality tests and the PSNR test results.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

• Read part 1 of Andy Quested's HD guide: The Hitchhiker's Guide to Encoding: Before we start
• Read part 2: The Hitchhiker's Guide to Encoding: Life, Encoders and Everything (Or a brief history of HD encoding)
• Part 3: The Hitchhiker's Guide to Encoding: So Many Tests, and Thanks for All the Recommendations (Or the BBC and the EBU)

BBC Research and Development is one of a diminishing number of European laboratories with the expertise and facilities to carry out testing for, and make contributions to the EBU's technical committee. I am extremely proud to work with the experts in BBC R&D and for the opportunity to contribute to the work they do.

Many of you have mentioned our current frame size and said:
"1440 Horizontal pixels is not HD"

To answer this I can point to several EBU documents that cover this point:

EBU - TECH 3328 Current Status of High Definition Television Delivery Technology (May 2008)
...In addition, the Sony HD-CAM and Panasonic DVCPROHD formats record only 1440 samples per line (with camera scanning at 1920 x 1080i/25). There is arguably no point in broadcasting material derived from this format at more than 1440 samples per line (although on the HD-SDI interfaces, a 1920x1080i/25 signal is carried).
A similar situation exists with the DVCPROHD format that horizontally sub samples (down-filters) the 1280 x 720p/50 format to 960 x 720p/50 (though on the HD-SDI interface, the signal is 1280x720p/50).

This document mentions 1440 and 960 as used in production. The HDCam tape format and some cameras that record to the DVCPro100 format use pre-filtering to reduce horizontal resolution before the signal is compressed. These cameras and formats are still widely used but are gradually being replaced by cameras and recorders that can compress the full 1920 or 1280 horizontal resolution.

Another document:

EBU - TECH 3333 EBU HDTV Receiver Requirements (March 2009)
The following image sampling structures shall be supported (see TS 101 154 V1.9.1, which defines further formats beyond those listed here).
1920 x 1080, interlaced, 25 frame/s (50 fields)
1920 x 1080, progressive, 25 frame/s
1440 x 1080, interlaced, 25 frame/s (50 fields)
1440 x 1080, progressive, 25 frame/s
1280 x 1080, interlaced, 25 frame/s (50 fields)
1280 x 1080, progressive, 25 frame/s
1280 x 720, progressive, 50 frame/s
1280 x 720, progressive, 25 frame/s

Tech 3333 is about receivers and shows 1440x1080 is an acceptable standard for high definition transmission.

It is worth pointing out that HD-Ready does not actually specify the horizontal resolution for an HD display. HD displays must have a minimum vertical resolution of 720 pixels and display 16:9 images correctly. There is no mention of the required number of horizontal pixels!

Several other posts have suggested the BBC is not meeting EBU standards for transmission with the current bit rate and have said:
"9.5Mbs is not in line with the EBU recommended high definition bit rates"

Again there are several EBU documents covering this point but the key thing to take account of is the date of the references. This document for example:

EBU Tech 3334 Accommodation of HDTV in the GE06 Plan (Feb 2009)
EBU tests of stand-alone MPEG-4 encoders of different vendors have suggested ^[5] the following minimum fixed bitrates in order achieve an HDTV image quality providing a significantly better quality perception compared to good quality SDTV (e.g. 6 Mbit/s MPEG-2) for a wide range, including critical content:

• For the 1080i/25 HDTV format and horizontal sub-sampling to 1440 samples a minimum bitrate of 12 Mbit/s is recommended
• For the 1080i/25 HDTV format and no horizontal sub-sampling a minimum bitrate of 12 - 14 Mbit/s is recommended
• For the 720p/50 HDTV format and no horizontal sub-sampling a minimum bitrate of 10 Mbit/s is recommended.

The key here is the ^[5] after "suggested". Reading the bibliography, point [5] refers to:
EBU Tech 3328 (Current Status of High Definition Television Delivery Technology)

Tech 3328 is dated May 2008, and was published long before we started tests on the new encoders for the BBC HD Channel.

Bit rates are also mentioned in an EBU presentation by Adi Kouadio (Asian Broadcasting Union symposium in March 2009).

Trends & implementations of HDTV Broadcasting
...Minimum (video) bit rate to provide HD quality (from EBU tests - BPN085-087):

• 720p/50 - 10Mbps
• 1080i/25 - 12-14Mbps...

EBU test documents BPN085 to BPN087 detail tests carried out on specific encoders. These tests were undertaken over the last two years but none refer to the encoder we are currently using.

Tests have demonstrated that at transmission bit rates, H264 encoders should deliver approximately a 2:1 efficiency over MPEG2 encoders. Depending on the manufacturer and their current stage of development good HD at 8-10Mbs is achievable now.

The BBC has made contributions to most of the documents mentioned above and many others besides. We are also involved in testing and trials for high definition production, contribution and transmission compression and were one of the earliest activists in the latest round of HD activity in the EBU.

The majority of the documents produced by the EBU are Technical Recommendations and are based on work done at a specific point in time by the members. After publication many groups continue to work on revisions and updates to the recommendations as the technology behind the encoders (in this case) continues to develop and improve. EBU Tech 3334 acknowledges that:

...with the expected future developments in video coding, it is assumed that HD fixed bit rate requirements will be reduced to 8-10 Mbit/s per programme. There will also be advances in the transmission system such as DVB-T2...

For some strange reason the encoder manufacturers don't all bring their latest offering to market at the same time! Two or three times a year they do show off concepts or the next generation prototypes at the main trade shows but products to the broadcasters arrive as and when they are stable and ready. This year for example, several manufactures were showing early versions of HD coding at 4-6Mbs (average not minimum).

One final point worth noting here is the life cycle of the hardware. As I mentioned earlier, the old encoders had reached the end of their life but the new encoders are at the beginning. They exploit more of the AVC toolset than the old encoders even though we are on version 1 of the firmware. We will continue to add improvements as and when they are available.

Tomorrow I want to concentrate on picture quality analysis and how we set up the encoder tests.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

• Read part 1 of Andy Quested's HD guide: The Hitchhiker's Guide to Encoding: Before we start
• Read part 2: The Hitchhiker's Guide to Encoding: Life, Encoders and Everything (Or a brief history of HD encoding)

EBU - TECH 3328 Current Status of High Definition Television Delivery Technology (May 2008)

... EBU investigations in 2005 showed that some MPEG-4 H.264/AVC hardware encoders did not show any bitrate advantage over MPEG-2 and in some cases even performed less well than MPEG-2 encoders. This situation improved by September 2006, and continued to improve in 2007 and 2008.

The original encoders ran at around 19Mbs and struggled to maintain the quality we wanted for a mixed genre channel.

Early in 2007 tests began on the new firmware. This was a substantial upgrade, introducing new software tools that helped the encoder cope with the wider range of programmes and programme styles the channel now had. It also allowed us to reduce the bit rate to 16Mbs with no noticeable loss of quality. The new version went live just before Wimbledon 2007 giving us an ideal opportunity to compare the image quality with the previous year's tournament. Direct comparisons between the 2006 and 2007 tournaments were very encouraging especially when matches went on late into the evening and camera gain (and therefore camera noise level) was higher than I would really like to see.

After the change I did a couple of blogs about TV set-up and about the different programme styles we were transmitting, particularly about the use of film motion (25p) and shallow depth of field (I will talk more about programme styles in a future post):

• The Scourge of SCART
• Picture Quality and Dolby ResearchPicture Quality and Dolby Research

The Angel Falls sequence was an early use of the helicopter set-up that gave the series such iconic shots but after seeing the rushes we were worried about what looked like interference on the tapes.

When the sequence first went out, the original encoder and software didn't resolve the noise but when the clip was used later, the noise was clearly visible.

Above: The Angel Fall sequence from Plant Earth from the programme tape.

Left: This is a blow up of the rocks in the picture above. The horizontal lines are noise on the original tape.



This leads me on to taking screen grabs or capturing still frames from the channel's transmission signal. It has been fascinating to see some of the stills grabbed off air. Still frames are an interesting analysis tool for picture quality and for making comparisons but they do have to be completely identical and the stills format used must either be uncompressed or the native file format of the coded frame to eliminate additional artefacts as a result of concatenation with the stills format itself.

This is an old, but useful article if you want to read more about concatenation and digital broadcasting:

http://www.ebu.ch/en/technical/trev/trev_271-dalton.pdf

Getting identical stills from an MPEG signal to use for quality comparison is not straight forward. Ideally the image should be the same to make comparisons easier but it is more important that the frame type matches. Typically MPEG 4 encodes frames as one of three types; I-Frames, P-Frames or B-Frames.

I-Frames are coded in isolation without reference to any other frame.

P- Frames (Predicted frames) are made up from the differences between the current frame and previous I or P frames, known as "reference" frames.

B-Frames (Bidirectional-predicted frames) are made up from differences between the current frame and preceding and following reference frames. Unlike MPEG2, reference frames for B-frames can be I, P or even B-frames themselves.

http://en.wikipedia.org/wiki/Video_compression_picture_types

A group of pictures may be made from a sequence of I, P and B frames starting with an I-Frame e.g. IBBPBBPBBPBB - IBBPBBPBBPBB etc. The exact order can be adapted to take account of shot changes and motion within a sequence. This grouping of frames is known colloquially as a GOP structure (Group of Pictures) - imaginative aren't we!

Although still frames are a useful tool for picture analysis, quality comparison using stills is best done with reference to the original source and using the appropriate I-Frames.

Looking through the stills that have been posted it has been really interesting to see quality of some of them, including the odd one or two that suggested the new encoder is better!

Tomorrow is a chance to look at some of the EBU documents on HD and the BBC's involvement with EBU Recommendations.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

Read part 1 of Andy Quested's HD guide: The Hitchhiker's Guide to Encoding: Before we start

Because there's a lot to say, I thought it would be better to serialise this blog over the next week with each episode covering a different topic. I will make no apologies for the titles and many thanks to the memory of Mr Adams.

Today: Opening and audio issues
Monday: HD encoder history
Tuesday: The EBU
Wednesday: PSNR and all that
Thursday: New encoder evaluation
Friday: Programme styles and closing thoughts

The BBC HD channel is two this weekend and in that short time we have developed one of the widest ranges of programmes you could find anywhere in a single HD package. No one is going to say it's all been perfect and in the past I have had to make a couple of very confessional blog posts when we've "just got it wrong", for example:

BBC HDTV: "The BBC's Bold Trial Of Reverse Karaoke!"
BBC HD: Olympics Opening Ceremony Surround Sound

We take every aspect of the channel very seriously and consider you to be a valuable asset when it comes to problem solving, trouble shooting or just "plain" talking! I'm sure Paul won't mind if I quote a section from Roly Keating's reply to his email:

"...Andy Quested has himself been considering the most recent exchanges on this subject, in consultation with the BBC's R&D teams, and will be posting further messages in response very shortly. Please consider my comments in conjunction with his: I hope that between us we can at least persuade you that we are paying the closest possible attention to the debates on this subject, and working constantly to keep standards as high as possible."

Before I start, it's worth saying encoder testing takes a long time because it's not just about picture quality. Before any testing starts a new encoder is checked for full compliance with the MPEG4 standards. Then it's tested for a considerable time to make sure it's compatible with existing set top boxes from different manufacturers and with the Sky platform. Then for audio compatibility (surround sound and stereo), for audio description, for subtitles and for interactive services.

Although we test the encoders on as many receivers as possible, we can only commit to a guarantee of compatibility with the supported platforms, so for satellite that's Sky and Freesat. I know several of you use DVB cards in PCs and Macs and have had problems. However there are so many different devices, options, versions and software variables, it's just not possible to check everything and therefore we cannot support these receivers properly. If you do use a DVB decoder card, have a look at this site for more help:
http://thegreenbutton.com/forums/t/73544.aspx?PageIndex=8

The next few days will give you a glimpse of some of the work done to maintain the quality of the BBC HD Channel by BBC Research & Development, our partners and to a lesser extent, me!

So to begin... Audio!

The Holes at the End of the Chain

Back in 2008, Rowan led a detailed overhaul of the HD Channel's audio system to make sure the new AV Sync test signal would really be sync when it arrived at your set top box!

We made sure the Channel's AV sync was within 5ms and we check it regularly to keep it there. The number of complaints about AV sync dropped to virtually zero after the signal was first shown and, with the exception of a couple of live programmes, it's been that way ever since. We've also completed thorough training sessions with all our main service suppliers to make sure they understand the issues of distribution and transmitting surround sound signals and metadata.

Recently though, we've had a couple of weeks where nothing seemed to work properly. It started with holes in the audio during Strictly Come Dancing. Your posts suggested they were on satellite and cable, so the investigation started in Television Centre. I was convinced it wasn't the right place to start, call it professional pride but we had gone through the chain pretty thoroughly, and as I said, it's checked regularly.

So I kept listening and a couple of days later I heard holes during the live Later with Jools and in the promo but this time they were much smaller. I asked our playout people to keep repeating the clip in the promo while we listened on every receiver we could find.

This proved the holes were only on the satellite output and we quickly tracked the problem to one of the HD Channel's "resilience" circuits. As soon as it was isolated, the holes stopped. We knew what had caused the problem but are still investigating why!

For 48 hours we had nice clean audio but just when I thought I could breath again and we had audio under control it promptly failed again, not once but five times!

As the first Electric Prom went to air the holes reappeared. I was watching at home so was on the phone in about 10 seconds. This time the play-out centre could hear them and they sounded like the surround signal was not synchronous with the video reference.

The Channel director went to emergency cut mode that removes all play-out processing, stopping the holes but not curing the problem. While we investigated, I decided we should use the reprocessing unit in the central switching area put in after the 2008 Eurovision Song Contest. This cleans up signals and modifies metadata if required, before transmission. It solved the problem for the next transmissions but we had not identified the cause. Timing issues are notoriously difficult to track down especially in mobile systems, but we will find it!

There was another audio problem during Thursday's prom. The programme started well but I had been warned it would be edited on site and it would be very tight to transmission, so tight in fact that the play-out machine hadn't finished caching the audio as the second half of the edit started to go out and guess what - holes in the audio. They were "real holes" that is no audio at all, and they could be clearly heard on BBC2 as well as the HD Channel, no consolation I know but we had to put this down to machine failure and unlikely to be repeated.

Back to Strictly and a live OB from Blackpool, not the sort of show you want to mix from the back end of an OB truck no matter how big! Due to running around all day and several accidents in front of me on the M40, I didn't get to a TV in time for the start and then it was BBC One only. It didn't take long for me to find out there were serious audio level problems on the HD Channel but what was worrying was no one in the broadcast chain could hear it, to me that shouts metadata!

Later that evening I listened to the programme at home and it sounded like the set top box was being over driven. Strictly has a pretty tough mix and if the metadata isn't right it will cause major problems with the dynamic range control in the set top box. By Tuesday morning, I had the tape from the OB and could see the metadata and knew what settings the sound team thought they were mixing to and, more importantly, what they actually set.

It was quite amazing hearing the difference when I switched between the two metadata settings. We have now gone through the metadata requirements for the programme and made sure they are fully implemented.

What next? The Remembrance Day service is a very important event and due to the complexity of the OB it was going to be in stereo. When the programme started I just couldn't believe I was hearing production talkback and what turned out to be Radio 4's sound. Sending a non-broadcast audio circuit, or another networks audio to air is just wrong and should never happen. The seriousness of this incident meant an immediate investigation.

On the day the links had been established in good time and a problem with the incoming talkback circuit was identified. The stream coming back from the OB contains several audio channels and communications circuits embedded into the video signal. This is quite usual and guarantees the audio and video paths are the same and therefore always sync. The stream contained the main audio, clean audio (no commentary), Radio 4 audio and communications. To help fix the talkback problem the audio channels were "shuffled" so talkback appeared in place of the main programme along with Radio 4 - this should never be done. During the final line up phase the talkback was muted and the Radio 4 audio at that point would have sounded identical to the TV audio. In other words the channel director would have had no idea anything was wrong.

Last - and it better be! Yes, Doctor Who should have been in 5.1!

Something the dubbing mixer pointed out quite forcefully on the Monday morning after transmission. Before he contacted me I had discovered the programme was only in stereo on the server but the delivered tape had the full surround audio. After a bit of investigation we discovered someone had routed one of the server ingest video tape players to another area and changed the audio replay options to a non-standard mode. When the machine was routed back to its normal role the audio set-up was not reset and the next programme in the stack for loading was Doctor Who.

Machine or circuit failure is one thing that we all have to accept happens occasionally and we apologise if it does. But I have made it clear we must stick to agreed practice and procedures because they are there to prevent errors. I hope the action we have taken will keep the audio on the channel running smoothly with no more mistakes.

In my next post I want to talk about the Channel's HD coding in general as an opener to the main encoder testing posts.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

Dear tagmclaren

Thanks for the comment.

After speaking to my colleagues in Research I have some bad news for you.

The Nyquist limit for the BBC HD Channel is 27.84375 MHz so there is nothing above that frequency.

As I said in the blog, if you have a 50" 1920x1080 display you will get the 4th (20MHz) but you will also get some (but not all) of the 5th (25MHz) grating. However the 6th is just not there! What you are seeing unfortunately is an alias that's folded back at around 26MHz.

Here is your picture with the original 25 and 30MHz gratings added - if you look at the off-air and original 25MHz grating you can see they line up but the 30MHz original does not match the off-air grating.

I am pleased you can see the 25MHz grating though!!

Andy Quested is Principal Technologist, HD, BBC Future Media & Technology

Many of you have been asking for a test signal to help line up your own HD TVs, we have been listening but it's taken a while to get it sorted.

From this week the HD promo has two test signals and I want to talk about how to find them and how to use them to line up and check your home systems. I also wanted to share a fascinating mathematical proof that some people (Heroes style) can change the flow of time!

As many of you have noticed BBC test card has been going out for a couple of weeks, this has now been joined by an Audio/Video Sync test signal. The test card seems to have been given the name "Test Card X" but not by us, it is in fact a modified high definition version of test card W (named because it was widescreen!) and for those interested in the history of test cards, there is an interesting "romp" through it here - it even includes the current incarnation!

The HD version uses the very famous picture of Carole (George Hersee's daughter) re-scanned in high definition and added to an HD version of Richard Russell's well known widescreen test card.

Now for the purists there's a bit of a disappointment coming. No, not the fact the test card's only there for 90 seconds every two hours! Talking of that, I was with a group of people looking at the promo last week when the test card came up - they all said "does this have to up for so long" and "what's that noise on the sound track" I did attempt to explain how much it was wanted but it just made things worse! I said I had wanted 5 minutes and many of the posts had asked for up to 30 minutes - at that point I felt like I came from another planet and decided to get on with other things! But there is a test card going out and I hope we can all celebrate its reappearance after many years!

The disappointment is a technical one. I am going to admit I have doctored the test card - much to the disgust of many of my Research colleagues. Why? Two reasons actually. A high quality test signal like the HD test card is a very valuable asset and unlike the SD transmission chain the HD one is quite good and quite capable in the being "purloined"! Already some of the posts on digital spy have already gone into great detail with the exact measurements of the card.

This version of the test card can be easily identified as it's the only version with the HD DOG logo at the bottom. Now, I want no DOG posts in this blog, I will ignore them as the DOG debate goes on elsewhere.

What have I done and how useful is this version of the test card? First, white level has been reduced so the peak white box is not 100% (level 235 or 0.7v). The super white spot is now 100% and the linearity of the grey scale is now slightly inaccurate. However no domestic displays have the level of adjustment we expect a broadcast monitor to have, so I this does not affect the usefulness of the test card to help you line up a "normal" TV. Also the colour bars are slightly lower in colour level. My apologies go out to people like Richard Russell and all the others who made these test charts possible - but this does protect the value of the work.

The second reason is to help protect screens from burn in. The full level test card will burn a screen in quite a short period so please heed this warning:

DO NOT leave the test card on screen for more than 2 minutes if your screen is less than three months old or more than 5 minutes on older screens. Make sure you go back to the promo for several minutes before using the test card again.

If you want more detail of the changes there is a very good post on Digital Spy. If you do have a broadcast style display at home it is quite easy to calculate the offsets to apply to a colorimeter to make sure the readings are correct.

Now for a bit of an explanation about the test card and how to use it to line up your TV, I have done this at home so can say it does work. But before you start to line your set up please take note of the following:

1. Make sure you have the user manual and know where the controls are.

2. Do not do this if you are unsure of any of the controls or there effect on your television picture.

3. It is best to do this in a darkened room, it doesn't need to be completely dark but if it's too bright or there is a lot of light falling on the screen the results will not be good.

4. Many modern flat screen televisions have presets for sound and picture. Write down which one you use so if you get lost you can always go back and start again.

5. If you have a PVR it would be a good idea to record the test card section of the promo. Most of the line up can be carried out on a freeze frame of the test card. If you do this please be mindful of the warning above about screen burn.

6. If your TV has it, change the picture settings mode to "manual" or the equivalent, so any inactive controls become active allowing you to change the settings on the TV.

7. Turn the sharpness setting to off or zero. If there are any picture enhancing options, make sure they are turned off or to zero (if you can). Remember, on some TVs the sharpness control has a centre zero allowing you to soften pictures - please don't do that!

So to start:

BRIGHTNESS AND CONTRAST

There is a GREY SCALE to the left of the picture on the test card. It's there to show the correct black and white levels of the picture. Broadcast displays have the ability to adjust the grey level independently so there is a linear grey scale between the black and white blocks. I am not going into how to use this here if you are interested start with this.

The top white block has two spots. As I said earlier, usually the white block is peak white with the right spot higher (super white) and the left spot slightly lower. On our test card, the levels are slightly reduced.

The bottom black bock has two spots, the right hand one is below black level (sub black) and the left is slightly above black. The modifications to the test card have not change these levels.

BRIGHTNESS

To set the brightness:

1. Turn your brightness control up until you can see both spots.

2. Turn the brightness down until the sub-black spot disappears but make sure you can still see the left slightly brighter spot.

CONTRAST

When a broadcast monitor is lined-up properly, we use a meter to check the white level however on a domestic television contrast is more a matter of personal choice and will be different on different types of display (LCD, Plasma, Projector etc.)

Adjust the contrast until you like the overall look of the test card while you are doing this, keep an eye on the spots in the top white block to make sure you can still see the left hand one. It doesn't matter if you cannot see the super white spot so don't worry if it's not there.

COLOUR

Again colour level is very much down to personal taste but most TVs have too much of it! Too much colour makes pictures look very odd. It will also make some colours bleed into each other or appear to move so the colour smears over the edge of the object - in other words someone wearing bright colours clothes may have the colour slightly off to one side! The best bit of the test card to use to set colour is the picture of Carole.

The centre of the test card has all you need to get the colour right. Carole's face should look natural and the primary colours in the picture (red dress and green and blue of the clown) should not be very bright. Colour is a subjective setting so just make sure you like it. Remember, if your colour setting was previously set very high you may not like the correct level until you get used to it!

One of the experts at BBC Research suggested another way to adjust colour level.

Get some Lee Lighting Filters No.181 Congo Blue and place it over the screen. This has the same effect as turning off the Green and Red leaving the Blue component of the picture. Looking at the colour bars around the edge to the test card, adjust the colour control until they all look the same brightness. There are some commercially available line up DVDs that use this method.

When you have adjusted the BRIGHTNESS, CONTRAST and COLOUR have a look at the promo again to see what you think. Watch it for some time so you get used to the new settings and see several different type of programme.

SHARPNESS

I have the sharpness control on my TV set to zero but some of you may want to add a little bit if the picture looks very soft.

To the right of the picture of Carole is a set of "frequency gratings" The frequencies are:

1. 5MHz 2. 10MHz 3. 15MHz 4. 20MHz 5. 25MHz 6. 30MHz

The BBC HD transmission system will pass frequencies 1 - 4. Most domestic displays will show 1-3 correctly but the 4th might not look quite right. A 50' 1080p display should be able to resolve the 4th grating satisfactorily.

PICTURE SIZE AND POSITION

Not all TVs offer menu setting that allow you to change picture size and position. Even if your TV does allow you to adjust size and position, it's not a good idea not to make anything but small changes unless you know what you are doing. Make a note of the current setting BEFORE you change anything!

Most displays lose a small amount of picture all round. This is called "overscan", it is perfectly normal and programmes have always been made taking this into account.

Some flat screens do have the option to either turn overscan off or reduce the picture size.

It is perfectly safe to use the "overscan off" option on you TV but you should not use the picture size controls for anything more than small changes.

The full test card should look like this, with the diamond points just touching the edge of the screen all the way round.

As a mater of interest, the cross on the Noughts and Crosses game is the centre of the picture!

You should now have a picture that looks fairly close to the one we see before transmission. Again watch some of the promo to get used to the new settings. Also if you have turned overscan off, you might want to look at some SD channels to make sure you don't see extra bit of the picture you don't like. You may see some white lines at the top of the screen on some News programmes for example. This happens when signals are brought back that don't fully meet the broadcast standards, but have to go to air too quickly (if not live) so it isn't possible to correct them.

AUDIO VIDEO SYNC

The second test signal is there to help you check and adjust audio/video synchronisation. AV sync has been the bane of my life ever since the test channel started. Remember we have rebuilt the HD Channel infrastructure round a service running 24 hours a day, 7 days a week so occasionally we have had to put new sections into service without being able to fully test them.

A couple of months ago, Rowan stated working with me to try and clear up our surround sound and AV sync problems. It has been a joy watching him dive headlong into the issues and some of you may have visited Rowan's blog to see how his work is progressing.

The last time I saw him, he was waist deep in diagrams of installations and programme signal routes to see if each video process had a suitable audio delay and each Dolby process had an appropriate video delay in circuit. The idea audio processing "takes time" is relatively new to television and we have to remind people to compensate for processing delay in the appropriate place so we can make sure everything is correct it if anything needs to be changed - I hope the message is getting through!

However even when we get it completely correct, some home setups can cause A/V sync problems of their own. The second test signal should help you check and adjust the sync timing of your AV system. This does not work on audio fed through the HDMI cable to the televisions own speakers. Any delays in that situation should be compensated for inside the TV. A/V sync is only adjustable when you use a AV systems connected by the optical/SPDIF output of a set top box or for AV amps that can us the HDMI output and have their own delay controls.

As some systems could have two ways to adjust AV sync - the set top boxes will have an audio delay option in the set up menus and good AV amplifiers may also have audio delay options, you need to start by setting all delays to zero. Again, please make a note of the settings BEFORE you start.

Why is there a need for A/V sync adjustment now? Most flat screen displays introduce a delay while they process the picture before it's displayed. Inside the TV the audio is delayed to match the processing delay but if you connect your set top box to an external audio system, the sound can be one or two frames ahead of the picture. In nature this is not normal and we can detect sound ahead of vision very quickly and it is "just not right"!

Our transmission system can also introduce delay to both audio and video signals. Some of the delay is obvious e.g. if we send the audio through a Dolby E decode/recode process, the sound is delayed by 2 frames so we must add a 2 frame video delay. Other process aren't so easy to check as the delay occurs inside a device that's processing audio and video together so the reason the A/V sync signal was not transmitted two weeks ago was to allow us time to test our whole system to make sure what we send you is actually in sync!

So it's time to introduce the BBC Research sync check signal...

The audio is actually two blocks of wood being banged once a second - nothing to beat the real thing! The video is made up from three components:

1. A travelling bar marked in frames starting 12 frames before the audio clap and going on for 12 more frames after. 2. Three Sync Flash Lines. 3. A sync "plunger" or "clapper bar" (acting like a clapper board)

Before I tell you how to use the signal, you might like to know what we did to make sure the signal itself was synchronised and the transmission system did not put it out of sync. We - or I should say Rowan - have been measuring the signal at every point in the chain to make sure it is as accurate as possible when it arrives at your set top box.

This is the task I set Rowan to about three weeks ago!

Why so complex? I needed to make sure the signal followed the routes many of our programmes do through post production and audio mixing, playout and transmission.

What is sync though? If you think about the speed of light vs. the speed of sound it's fairly obvious that sound arrives a lot later that the image of the "thing" making it. A rough rule is audio takes slightly less than 3ms to travel 1 metre so if you sit 2 and a half metres from your TV the audio takes nearly 7.5ms to reach you - nearly a quarter of a frame.

The effect of AV sync has been measured and tested quite extensively by the international broadcast standards bodies and we usually work to a tolerance of +20ms to -40ms (+half to - 1 frame) for a programme delivered to the BBC. This tolerance has been well tested in SD but there has not been enough work done to see if it's still OK in HD. To make sure there were no major surprises we have tightened the delivery specification to +10ms to -20ms while we do more tests.

In one of my previous blogs I explained that during the trial we found sync varied during a programme, especially live programmes, depending on how hard some of the early equipment was working. Now we are a lot more stable and have had a chance to go through the system from end to end to make sure it's sync. We have just finished testing the chain with an "off air" test of the signal and have a timing error of 0.86ms or 0.0125 of a frame!

Those of you who work or have worked in the business know the phrase "it's alright leaving me". A translation of that is "I'm OK, it's your problem"! To make sure you can use the signal to check your home system, we have gone one step further and made sure the signal is "alright arriving at you", not just "alright leaving me".

The final measurements of the off-air signal were made by looking at the digital signal from the transmitter - or in this case, the one received from the satellite.

This is what the test signal looks like in the transport stream from the satellite.

Rowan has a more detailed explanation of how and why we did this in his blog.

Part 1: Don't forget the kitchen sync

Part 2: Testing the test

How do you use the signal to check audio/video (AV) sync?

Remember this only works if you are connected to an external AV system. Check the audio delay setting in the set top box is 0ms and if your AV amplifier has a delay check that is set to zero.

Method 1

Look at the travelling bar at a point before the centre - look say at 10 on the left of screen. Listen to the clap and see if you think the bar has passed this point before you hear the sound. You might want to mask the right side with a bit of paper or put your finger on the number to help.

If the audio seems to happen after the bar has passed, move on a number and repeat until you think the audio and the point the bar passes your maker coincide. The sync point could be between two numbers but most devices only make corrections in half frame increments you will have to decide if you think it's closer to a number or closer to a half way position.

You should still be on the "video late" side of the zero mark. Read the number (or closest half number) and multiply by 40.

If your number is 3, the audio delay you need is 3 x 40 = 120ms. If your number is 1 and a half, the audio delay you need is 1.5 x 40 = 60ms

If your sync point is on the right hand side of the zero mark, I'm sorry to say there is nothing we can do to help. Before panicking - check you have no audio delay set then wait for the test signal to come round again.

Method 2

For those who like a challenge there is an electronic method. The white lines flash for 1 frame at the start of the audio waveform.

I am sure you can think of many ways to use this information to measure AV sync accurately, but here is a simple option for all of you with a dual beam oscilloscope, a photocell and a microphone lying around!

If you place the photocell over the top sync flash line and the microphone on one of your front speakers, connect them to separate input of the scope (with any amplification devices needed to boost or power them) you will get two spikes. Make sure your scope is configured to display the two traces at the same time and measure the difference in ms. Apply this delay to either the set top box or the AV system. If you have a very good AV system you may be able to get this exactly right instead of the nearest 20ms.

If you can't decide between two settings, it is always better to make the audio slightly late than have it in front of the pictures.

Why are there three sets of white flashes? The top line is the reference line, i.e. 0ms A/V offset when measured on an HD CRT. However sometime it is difficult to accurately measure the very top of active picture, possibly because the TV's casing gets in the way of the photocell. The second line is 1ms later (as measured on a CRT) and is usually easier to get an accurate reading from. The third line is in the centre of the active picture so should read 10ms A/V offset on a CRT.

On the various LCD and plasma displays we have tried this on, some show a difference between the three lines and some don't - not much help to you, but I would go for the second line if you can and minimise the delay there!

Please let me know how you get on.

How to record the BBC HD test signals

The HD test card is just over 1 hour into the promo and the AV sync signal is 50 minutes later. To record both signals, check the time the last programme finishes and add 1 hour. So if the last programme ends at 01:30 set your PVR to record from 02:25 to 02:45 for the test card and 03:15 to 03:35 for the AV sync signal.

My last thoughts this time are around phenomena I have discovered that manifests itself around my daughter and what used to be my phone. When she is on the phone I have discovered time slows down!

How do I know this? Simple maths!

Let's take a telephone billing period - call it Tm. A Tm can only have 4 values 28, 29 (every fourth year) 30 or 31. Each Tm is made up from telephone charging units, let's call them Tu.

Each Tu is charged at various rates but I am going to use the maximum UK rate £UK (premium numbers etc are barred).

So my maximum phone bill can only be Tu x £UK. As the Tm can have four values the total bill can vary between each Tm. So why have my bills been consistently two to three times this amount?

My thought are, as the rate £UK is fixed in any Tm (but in general is always rising) and the periods of charging Tm are fixed to one of four values the only thing that can change is the Tu! As this is measured in time, I can only conclude time around my daughter changes as she uses the phone. Oh! I forgot to say, we are on the free weekends and evenings tariff so this time dilation effects occurs in the brief period between the end of school and 18:00!

I think Douglas Adams spotted this change to the laws of maths in the Hitch Hikers Guide - other explanations welcome!

Have a very merry, in sync and well lined-up Christmas and we will speak again in the New Year.

Andy

P.S. When she is on the phone to us each Tu is very long period of time of course.

Andy Quested is Principal Technologist, HD, BBC Future Media and Technology.

Not all of them are negative in fact many of them are very positive.

Whenever I see negative technical comments I do always look at the issues.

Only last week hobwell spotted the 5.1/2.0 switching had stopped some time during the evening. I hadn't watched the stereo programme concerned and I didn't see the comment until 18:30 the following day but I watched the channel start and as I knew the first programme was stereo I was on the phone to the duty engineers in a few minutes. We had the problem fixed, put out an on air apology very quickly and I posted some comments in Danielle's DOG Blog!

(Dolby labs picture from deltamike on flickr)

Other comments on quality do take some investigation.

As a slight change to my usual commentary I wanted to share some of a reply I gave recently to a more general complaint about picture quality on the BBC HD channel.

It also allows me to introduce Rowan de Pomerai, a BBC Research graduate who is working with me for the next few months on a project to improve Dolby E sound on BBC HD.

Rowan has his own blog so you can keep up to date with his project and his thoughts there!

Here is an edited version of the response to the complaint:

"Thank you for your comments about the quality of programmes on the BBC HD Channel...

I want to assure you we do take picture quality very seriously and use a rigorous quality check before any programme delivered on tape is transmitted. Whenever we can we also work with studio and outside broadcast companies to make sure live programmes meet the highest standard too.

You mentioned Amazon was "colour merging". I am not sure what you mean by this but I have not seen anything similar on my home display. I have spotted one other post that compares Amazon to the Sky 1 series "Ross Kemp on..." saying the latter has much higher picture quality.

I have watched both series in HD and must say they are similar in several respects, they both have very good HD content mixed with standard definition material and some sequences that have obviously been shot under very challenging conditions.

I agree some of the low light and night sequences in Amazon are not up to the standard of the majority of the programme.

Programmes like Amazon will always have sequences where conditions mean no matter how good the broadcast technology, domestic cameras that are always much smaller and less conspicuous will be used to reduce risk to the crew with a resulting loss in picture quality.

We are always looking at new technologies that will improve the picture quality in challenging environments but it will take some time before it is uniformly high. In the mean time programmes must limit the amount of standard definition or low quality high definition to 25% of their duration.

Your comment about the Tudors is one I cannot understand or agree with. I still have the series on my PVR and watched quite a bit before writing this. The Tudors is extremely well shot, has very little if any video noise and is beautifully colour balanced.

If this programme looks poor on your television could I ask you to re-check your settings? I would recommend turning the sharpness setting to zero, not to use any of the preset picture modes and to turn off, or reduce to zero all picture enhancement options. I would also recommend turning the contrast setting of your set top box to medium or low before adjusting the brightness and contrast on your display.

If you are looking at some of the chat rooms commenting on the quality of the BBC's HD Channel you will have seen threads discussing our transmission bit rate. The channel's bit rate has remained constant at just over 16Mbs since early last year. I don't often recommend external websites but you might find this one interesting.

I am always watching the channel and do make recommendations to any programme that has variable or substandard sequences to see if we can improve it. All programmes have to meet the technical standards of the channel before they can be transmitted but occasionally even we are caught out by a problem that does not come to light until the actual transmission.

The quality of some several recent programmes has been outstanding. The Goldfrapp Electric Prom was one of the best I've seen, Tess Of The Durbervilles was beautifully shot and from the sections I've seen Little Dorrit should be stunning.

The range of programmes made in high definition will continue to increase over the next few years and we will explore what works and what is just not worth it until high definition is the normal mode of operation.

Many of our high definition programmes use the 25 frame progressive standard (film style). I know some people do not like this and think it degrades the resolution of the picture, while others think it contributes to the quality and style of the programme. This mode does actually have more resolution than the 25 frame interlace standard. Amazon, Silent Witness, Tess, in fact virtually all drama, Natural History and many documentaries use this standard.

Cranford, Silent Witness (right), Tess and other dramas are also using the latest large image format cameras. Theses cameras use a single image sensor that is about the same size as a 16:9 35mm film frame and gives the image a very shallow depth of field. A shallow depth of field will put all but the key subject out of focus and allows a director to use focus as a story telling tool. Again some people think high definition pictures should be pin sharp from the nose of a person in close-up to the trees on the horizon, others find all this visual information distracting and a drama director will use focus to point you to the action they want you to watch.

Here lies another point of confusion; sharpness is not the same as resolution. A picture can be very sharp but contain very little detail. This is especially true in standard definition where electronic sharpening is added in cameras to make the image seem clearer. We do not encourage the use of electronic sharpening in high definition cameras and prefer images to look more natural.

I am currently working on an issue we (and other broadcasters) are having with the Dolby E signal that's used move up to eight channels of audio in the space of two. I have a research graduate from BBC Research working with me, this is very exciting as it is an opportunity not only to deal with problems but to delve deeply into the underlying technology and maybe make improvements that all broadcasters can benefit from. Rowan has his own blog and he will be posting comments on the project when he can - here is the latest.

To finish, there are many comments praising the quality of the HD Channel's quality. I am also aware of the many threads berating the channel's lack of quality, I do read them all and try to address some of the concerns in my blogs and as they come up.

If you would like to read more about BBC Research you can find them here

Yours sincerely...."

As I said at the begining of the letter we do take comments seriously and try to address complaints and enquiries. I hope this sparks off a lively debate and look forward to your thoughts on Rowan's work.

Andy Quested is Principal Technologist, HD, BBC FM&T.

We have taken some time to look at a lot of the issues bought up by you and a few of the things I wanted to investigate myself.

Audio

The big issue of 5.1 / 2.0 switching. Thanks to everyone commenting on this blog and elsewhere to give feedback while we were trying switching options. It's now working, but not as well as we would like - for example, we still have more work to do before we can move between stereo and surround programmes without having to put silence between them.

When things are changed, then no matter how well you plan there's always something that tries to catch you out. While we where upgrading some of the Dolby equipment, we came across an "interesting" problem - fortunately, it didn't go to air!

Some of you may know that we use Dolby E to move surround sound and metadata through the playout area (and if you read my post about the Eurovision Song Contest, you will know how important the metadata are!).

Dolby E is a frame-based data stream - in other words, it comes in packets timed to the video signal. If any packet is corrupt or mistimed, the whole frame is invalid. Some devices can deal with this, but others don't recognise the corrupted package for what it is and think it's a chunk of PCM audio. A Dolby E packet contains a lot of data and if any device treats it as digital audio, it comes out as a very loud "splat". Fortunately for your ears, it's only around 40ms long - this is, however, somewhat unfortunate for your speakers!

The work to get surround and stereo to work together is still going on and I will update whenever we have more to report.

Picture quality

Danielle talked about some of the quality issues you reported on Friday Night With Jonathan Ross in her What Works Best? post. We have looked at the whole studio, post-production and transmission chain to see what was causing the problems.

There were a couple of processes that made some of the cameras quite noisy: these have now been addressed and last Friday's show (October 3rd) looked a lot better, but we are still trying to find ways to improve the quality across all programmes.

Picture quality issues bring me to another story. I'm sure you are getting used these stories from my previous posting about the thoughts of my daughter - who, by the way, is now officially a stroppy teenager!

Actually, this is about a trip to pick her up from one of her school friends' and getting invited in for a coffee while I waited. "You work for the BBC" was the opening from the other dad - "do you have anything to do with this high definition stuff?". My answer was a bit muted, thinking, "what's next?"! He then said he didn't see what all the hype was about: the pictures were a bit better but nothing to get excited about.

I asked the usual questions about type of TV and how he received HD. As expected, it was a new 50" plasma - quite a nice one actually, and a Sky HD box with the full subscription package. "Come and have a look" - so, shoes off, coffee mug in hand, we go through.

He turned the telly on and, after wincing from the volume of one of the music channels left on by his daughter, we went to the BBC HD channel, which looks - to put it politely - "crap".

Image by Joe Anderson on Flickr under a Creative Commons licence

At this point I had to put down the coffee mug and admit to my job title. The diagnosis took about five seconds - so, trying to look very professional, I reached round the back of the TV and pulled out the SCART lead. After the usual bumps and flashes, the pictures re-established themselves looking like the high definition I would expect.

There was a short "Ah!" and we went back to coffee and chat about the length of time two teenage girls can spend on a phone after they've only just been at school with each other!

This was not the first instant repair I've done and I doubt it will be the last. One of the things I'm always doing is taking TVs out of "Vivid" or "Dynamic" or some other mode that makes the screen hot enough to do toast on. Others changes include setting the contrast and brightness levels to something resembling normal and most important, turning the sharpness down to zero and finally setting all the picture enhancing modes to "off". That usually results in reasonable pictures on any telly and tends to make HD look like HD!

I'm not sure what to do about the SCART lead. It seems to be the biggest single quality filter in the chain, especially as the default setting for the SCART output on many set-top boxes is PAL. My recommendation is not to have them at all, but there are still a lot of devices out there that seem to need them.

Over the next few weeks, we are looking at some new cameras and field recorders: we will keep looking at picture quality issues as they come up and talking to Dolby about a better way to mix 5.1 and 2.0 programmes - please keep letting me know how it sounds at home. I have a Sony AV system, but I need feedback from as many systems as possible.

Also, if you know anyone with a SCART lead connected to their HD set-top box, either pull it out or make sure that the SCART switching menu option is set to "OFF"!

Andy Quested is Principal Technologist, BBC Future Media & Technology.

So why wasn't the opening of the Olympics on BBC HD television in surround sound?

We had been having problems with the Dolby E stream on the line back from Beijing.

You have to remember we are sending back many video, audio and data streams from the games for all the interactive outputs, so it is a very busy set of circuits - but all seemed well just before the opening.

During a final rehearsal about 20 minuets before transmission, Red Bee noticed that the surround sound kept "popping"; meanwhile in the Television Centre switching area, they heard one or two pops - but not at the same time as Red Bee.

The problem was looked into at both ends and at about ten minutes before transmission, there were only a couple of options left - as these required a reboot of a large chunk of the system, you would naturally ask a couple of questions:

• 1. would it cure the problem? and
• 2. how long would it take?

As the answers were "maybe" and "more than ten minutes", I think the next step was obvious.

In my last post about the Eurovision Song Contest, I said we were putting in failsafe systems so that we could at least keep the high definition pictures if we had to use the stereo audio. Well, as promised, the infrastructure is in and the "failsafe" worked. (Photo of BBC truck at Helsinki World Athletic Championship in 2006 from timo_sw on Flickr)

We tried a couple of times during the opening to go back to what looked like a stable surround signal, but along would come another "bump" and in the end it stayed stereo.

Overnight (Beijing time) the systems engineers worked backwards through the chain to find the cause and the report back was during a check of a rack of equipment it stopped - that's it so far.

The next programme went on air in surround and it's been there ever since.

We are still looking into the issue to see if it was equipment failure, a problem with the link back from Beijing or a problem with the incoming audio from the host. Maybe I'll write another full post later!

Just for information, the host broadcaster is distributing 4.0 audio to all broadcasters taking surround. The sections from tape, archive material and the studio are in stereo.

Andy Quested is Principal Technologist, BBC Future Media & Technology.

I wasn't joking. It's been an amazing two years, with many successes and a few disasters!

Top of the list for "must try harder", though, is audio - with surround sound causing more problems than anything else. I couldn't decide what to call this post - I started out with: "Surround Sound is easy... ...to get wrong!"

But then there was an incident with the Eurovision Song Contest and I had a different idea. So, welcome to:

The BBC's Bold Trial Of Reverse Karaoke!
(Or, We Send The Vocals; You Provide The Accompaniment)

I'm sure that, like me, you've been to at least one Eurovision Song Contest party where you either had to dress up as one of the qualifiers, take a dish or bottle of the country you've been allocated, or been given a song sheet so you can sing along. This year, due to a technical fault, we had the ideal opportunity to try something new, so for around half an hour we sent the vocals for everyone to gather round the telly and provide their own instrumental accompaniment! From the comments in the many HD forums, I gather not many people took the opportunity, oh! well.

A couple of weeks ago, I was speaking to the technical director of Discovery: he said that over 75% of their quality control failures are due to surround sound problems.

Then you realise that Discovery are at least five years further down the HD road than us and they don't do that many live programmes! No excuse for getting it wrong, but it does put it into perspective.

So why and how does it go wrong and why isn't everything in surround?

It's worth remembering that well over 95% of the audience of any live programme is watching in standard definition with stereo sound and the standard definition audience must get the best possible service. This means, however, that we've just multiplied the problems by two.

On the day of the Song Contest, I had been providing my usual daughter taxi service and arrived home around 20:15. It did take a few moments to realise that something was wrong and that there should be a least a bit of music to go with the vocals.

My AV amp was saying 1.1 and this was a bit of a surprise because the transmission chain is supposed to be locked to 5.1 at the moment. This is another area for forum discussion and the BBC has been the subject of much derision, accusations that we don't know what we're doing and "what would Dolby say?" statements. So before going on with the Song Contest story, I'll give a bit of an explanation.

The HD channel only transmits one audio stream, and this is encoded either as 2.0 for stereo or 5.1 for surround. When we transmit a surround sound programme, the set-top box does a down-mix to stereo for anyone who listens via the analogue outputs or HDMI via the TV speakers. We send metadata in the audio stream to control the mixdown. It's worth remembering the metadata as they will come back to haunt us later!

When the trial service launched, we only had a few surround sound programmes - but I noticed, as we switched between 2.0 and 5.1, there were a few clicks on the audio when I listened at home. We tracked these for a while to try and find out what was happening, but they weren't serious and I initially put it down to my cheap AV system. Also there were no comments from the audience and even DigitalSpy was quiet. Anyway, we had bigger audio issues to deal with at the time.

As we started to get more surround sound programmes and were switching between live, pre-recorded, surround and stereo in every direction, the clicks began to get a bit more annoying and there were a few forum comments, so now I knew others were hearing them too. We started doing some investigation and I even recorded every junction on my home set-top box over a weekend - sad or what? I had to programme it all manually as the guys at Sky wouldn't do an upgrade to allow me to record every programme junction, but not the programmes - advertisers' dream option, I thought!

Connecting my box to a few home cinema systems produced some very interesting results. Every box did something different - the more expensive ones actually muted the output for 2-3 seconds over each mode change; this was something we just couldn't tolerate. At the time, we were talking to Dolby about the problem, trying to find out whether our switching technology was at fault, but it looked like this was a home cinema system issue.

Dolby has also done some investigation and there seems to be a problem with the way some home cinema systems switch between surround modes. It's very similar to the aspect ratio switching issues we had in the early days of widescreen, i.e. unpredictable and variable between manufacturer and model. Dolby kindly sent us a release that explains the issues in more detail and we are looking at possible solutions to the problem. However, in the meantime we are staying locked to 5.1 - well, that's what I thought until the Song Contest came along!

I said earlier, I was distracted by other audio problems. Domestic audio systems use something called Dolby Digital or Dolby D to receive and decode multi-channel audio. We use another Dolby format, Dolby E, to carry up to 8 audio tracks in the space of a stereo pair. It's a very useful compression system, especially for outside broadcasts where there's limited capacity on the link back to BBC Television Centre. The Dolby E stream also carries all the metadata the sound supervisor has added to control levels and the stereo down mix. We can pass this onto the Dolby D encoder and then on to the decoder in a Dolby home cinema system or set-top box.

Audio signals are usually embedded into the video stream now, and Dolby E uses virtually every available bit in a PCM stereo pair - sending any measuring devise straight into the red zone! Because of this, every device we use to either pass or process the signal has to be properly aligned, timed and to carry the signal as specified by the standards bodies.

During the first summer of the trial, we had been trying surround sound on different events - not only to get experience capturing, but also to try moving the signal around our infrastructure. Wimbledon had been a great success, and we were looking forward to the Last Night Of The Proms being the climax to a summer of live HD.

The way we get signals back to TVC is now critical not just for the quality of the video but also for the successful delivery of surround audio. During Wimbledon, we used a fibre link and, if you remember my last post, our issues were all about getting the fibre into the switching centre and were nothing to do with the quality of the signal it carried! To get signals back from the Royal Albert Hall, we used a microwave link to the PO Tower and then our permanent connections back to TVC. Not much of a problem - we've been doing it in SD for years and it only needs a few more Mbs for the video and an additional Dolby E stream for the surround sound. What we discovered during tests, though, was that the video and the stereo went in and popped out the other end with no problems but the Dolby stream just didn't go anywhere.

It was during these tests my Sky programme guide dropped through the letterbox at home, and it was advertising (on the same night as Last Night Of The Proms, no less), a live HD concert by a beat combo led by someone called "Robbie Williams". This was to come from Manchester, if I remember correctly, and I would have laid money that it would be in surround and the whole package would come down to Osterley via a satellite link.

After a few phone calls and a couple of beers, I discover that just like us, the Dolby E stream was going into Sky's sat link and resolutely refusing to come out at the other end! Both of us had identified the broadcast MPEG2 encoder as the culprit but we were not getting anywhere with a solution. Needless to say, we pushed very hard and a few days before the Saturday night, an updated version of the firmware arrived and the surround burst out of the other end of the chain.

By now (I hope), you have forgotten what this post is all about! But just in case you still want an explanation about the Song Contest... At 20:20, I was on the phone to the duty engineers asking why I hadn't been told we were doing a karaoke trial! They were very well aware of what was going on and were desperately trying to find the problem, but they did put me through to the HD network director and we decided to go immediately to an up-converted BBC ONE feed - this gave us properly mixed stereo audio, but of course we lost the HD. I said I would call the engineers back in ten minutes to see how the fault-finding was going. [Photo by johnthurmon Flickr]

Now, the next question you want to know is: "Why couldn't you take the HD pictures with stereo sound?". For that, you'll need to read on...

At 21:00, I was talking to the Red Bee duty engineers who had traced the audio back through the chain, but could find nothing wrong. It is worth a bit of time to tell you how the Eurovision Song Contest was passed through the chain from the venue to you.

The UK is a voting member so we have to feed ourselves back to the venue for the voting section of the programme as well as mixing Terry into the main clean feed coming in. To do this, we use a studio gallery. The engineers had traced the audio all the way back to the studio and listened to the direct feed from the venue, and at every point it was all there and correctly mixed. Everyone at every monitoring point could hear perfectly mixed surround sound and because of this, everyone was convinced it was a problem with the Dolby-E-to-Dolby-D conversion - the very last process in the chain. Meanwhile, the studio team (the very first point in the chain) was sure that all the settings were correct and that they could decode the Dolby E properly to prove it - but the Dolby D encoders were telling us we were sending 1.1 and nothing we could say would convince them otherwise!

Time (and contestants) were passing fast, so it was time for a change of tack and I am not one to believe the obvious - especially when two devices are saying something different to everyone else!

Earlier, I asked you to remember metadata, and by now I was thinking metadata not audio. I went back through the chain, but this time asked about the metadata that were nothing to do with operations (eg mix-down settings, dial-norm, etc).

I was getting grief all through this, as my daughter had now decided the Song Contest was not on her viewing list for a Saturday night and how she was to call several boys she had promised to call back if I was on the phone and she had to use the landline as I had deliberately let her mobile credit run out!

Anyway, and despite this going on in the other ear, several areas reported an inconsistency in some of the set up metadata readings but were not sure if it was a fault or a misreading as the results were different when monitored in different areas.

Before we did anything else, I needed to know what the actual metadata settings were leaving TVC and I also need to check lip sync because each Dolby E process delays the audio by 1 frame and we have to compensate by delaying the stereo and video to match. We had been checking and changing so much, by now there was no guarantee even if we did correct the fault anything useful would come out at home!

Lip sync is probably the second biggest issue we've had during the last two years. Each device or process that a signal passes through introduces some delay from a few lines or microseconds up to several frames. This is one of the reasons embedded audio is so useful but even embedders and de-embedders introduce delay.

From the comments on some of the forums, you would think we had people sitting back watching out-of-sync feeds and doing nothing about it. We do take sync very seriously and every sync complaint is always investigated. We have an off-air recording of the HD channel so that we can go back and check if we need to. But what is sync really?

A while ago, C4, five, ITV, Sky, Virgin and the BBC met and agreed how we would define sync. "Easy", you say - "lips and voice should match!". Balls should hit rackets; boots should hit footballs etc at the right time! Not quite as easy if one of your signals is encoded and it takes one frame to decode it to see whether it's sync. Also, in a chain that has known video and audio delays, do you add all these together and compensate at the front of the chain, or at the back? Or, as we decided, do you insist that all audio is sync with the video as it leaves each area, no matter how it's processed?

We define sync as "in sync encoded" for Dolby E signals, so if you cut an embedded stream on a frame boundary, the Dolby E and the PCM stereo will cut in exactly the same place in the dialogue. Each area is responsible for correctly aligning signals to meet this requirement, including any material recorded to or played back from tape.

One other thing we've noticed is sync drifts in some devices depending on how hard the device is working. In a complex chain, there could be ten or more processing units and if they're all working flat out, a signal path that was in sync during line up can suddenly be noticeably out of sync. Also, it's worth noting that you probably have to include how hard the home set-top box is working too when looking at end to end AV sync issues!

The EBU and other standards bodies are looking into sync specifications to see if the current standards are good enough for HD. We hope to see a tightening of the AV tolerance in all equipment soon.

Back to the Song Contest! To check exactly what our current state was, I asked the network director to switch back to the HD output so I could see what my AV amp did. That's why some of the forums reported a few seconds HD with mono audio! The metadata were still saying 1.1 on one of my boxes and 1.0 on another (please don't ask why I have two).

Now I had someone in the studio looking at the main Dolby E encoder, I was looking at a set up guide on the Dolby website (no, I don't have manuals at home) and I was talking through the settings I wanted to check. I really wanted them to check what the Dolby D metadata setting was, but no one could find it in the menu. Then one engineer asked if it could be the setting "AC3 metadata" - same thing I said, "What's it set to?". "Disable", was the reply - a quick change to "enable" and a phone chase back up the chain where everyone reported consistent metadata reading the Dolby D encoder said 5.1. The rest was easy, but the paperwork was hell.

So why did it ever get to air, what was wrong and why couldn't we do HD video with stereo audio?

Well, to start with, there was nothing wrong with the signal or the metadata and because of that the system did exactly what it as told to do - what was that? Simply put, we had told the system to ignore everything we told it to do! The embedded audio and metadata were not faulty, so the very simple system we put in place for the trial would not allow a non-standard change.

During the trial, we had to put in place a simple setup, but it was very well engineered to do a very simple job without intervention. Now we've been given permission for the HD Channel, we can spend the money needed to upgrade the system and to put in place all of the safeguards, monitoring and protection that a full channel requires. This has been going on around the "live" channel, and one of my first thoughts had been that some part of the upgrade was interfering with the main output.

The target for completion (including testing and acceptance) of the new infrastructure is the opening of the Olympics in August, which I gather is 8:08:08 pm on the 8/08/08! But we have just about got everything in place now, so from now on, we have the same type of transmission resilience in the chain as any other BBC channel. We also have the capacity to carry out the testing and trial we need to do whilst still on air.

It's my sincere hope that we are at the end of the first stage of our HD migration and that we can start to consolidate and to improve. So maybe this is not the sort of post you expected, but I hope it's given a bit more of the background of what we are up to and I would welcome comment or suggestions for what to do next.

Andy Quested is Principal Technologist, BBC Future Media & Technology

At the start of May 2006, there was no UK HD and by the World Cup two months later, BSkyB and BBC HD channels were up and running. Now there are 18 HD channels in the UK, with Freesat offering two free-to-air HD Channels with no need for a subscription.

Now I think the time is right to talk about the history of the BBC HD channel (technically, that is) by winding back to 2005 and sharing some of the stories of the unsung heroes who put together the technology. For anyone who wants to know more about the nuts and bolts of the HD standards have a look at the EBU document Tech 329E. I can recommend it as bedtime reading!

At the end of 2005, the BBC had just announced its intention to run an HD trial on the three broadcast platforms which was an interesting technical challenge to say the least. Sky was working on its HD satellite set-top box and Virgin had its V+ box option, but for digital terrestrial we had to start from scratch. DTT spectrum is very tight and a national trial was out of the question, but we did manage to negotiate permission to use one of the channels reserved for use after analogue switchoff in the London area from the Crystal Palace transmitter. The BBC HD channel was allowed to use channel 31 that sits between the C4 and BBC2 analogue services. It had to use relatively low power and it was monitored carefully to make sure there was no interference.

So, now we had the ability to transmit HD - but what was going to receive it?

I can only image the conversations John Zubrzycki (BBC Research) had with set-top box manufactures in late December:

John: We are going to do a trial of HD on digital terrestrial and need to develop a set top box.

John: No, this June - in time for the World Cup.

That was probably followed by the sound of the phone going down!

Picture: Richard Salmon testing set top boxes and televisions

However, two manufactures rose to the challenge and by May, BBC R&D Kingswood Warren was overrun with HD boxes and televisions being tested. Richard Salmon reckons he watched nearly a week's worth of television and test signals every time there was a software update - and there were many, many updates.

The boxes were finally delivered to the trial participants in time for kickoff of the World Cup.

Picture: Just a few of the HD Ready TVs under test!

Picture: The Humax and ADB Terrestrial HD set-top boxes under test

Moving on to programmes, one of my jobs before we had an HD service was to help production teams deliver their programmes in HD to some of our major co-producers. Very early on, we discovered that no-one would accept Super 16mm film as an HD format. I did a lot of work with post-production facilities to try to come up with a way to make it possible, but there was no compromise: Super 16 was just not HD.

So when our own channel came up, I thought we should have another go. After all, we would be using MPEG4 not MPEG2 (like our US co-producers), we would have a high enough bit rate and we could look at drama as well as documentaries.

I think I spent two solid weeks looking at different trials, transfer routes and transmission simulations before I had to tell Seetha Kumar (Head HDTV) that about 25% of the programmes we thought we could transmit couldn't be called HD. The problem was that the MPEG4 transmission system did a very good job of removing all the grain from the film but after that the picture looked soft and there was no way we could call them high definition. It was a tough time and we are still working on the problem.

Still, the HD Channel would have the World Cup and Wimbledon to open, followed closely by the Proms - and though they were themselves major challenges, if we could get the pictures back, they would be a great launch for HD.

That brings me to the engineering planning for Wimbledon (2006). We could not cover the whole Wimbledon site in HD but we could do Centre and No.1 courts - and, even better, we could do it with surround sound. Everything was going well until we discovered that the fibre which carried the 100Mbs link from site back to Television Centre popped up in a room on the wrong side of a wall! There was plenty of standard definition connectivity between the room and the main switching matrix - but no HD.

Someone suggested that we could down convert the pictures to standard definition and route them through the up convert for transmission - I'm not sure what his job is now! The discussion went round and round for a while trying to resolve the problem until the OB Engineering Manager suggested their solution would be a power drill and a large hammer! I'm not sure that everyone around the table appreciated it, but it did focus minds to come up with a reasonable solution!

The guys in the central apparatus area are always innovative when it comes to providing a quick and temporary solution. I found out exactly what "quick and temporary" meant when I went to see the first test pictures arriving from the World Cup site. All the original World Cup planning had been done when there were no thought of HD transmission and everyone had worked incredibly hard to move the whole operation to HD.

For the BBC coverage, this was made more complex because we had the main studio in Berlin while the International Broadcast Centre (IBC) was in Munich. The first shot we saw was a rather stunning view of the Brandenburg Gate through the glass wall of the studio and it was really good to see everything working. After a while, I was shown some of the equipment recently installed to route HD to Red Bee for transmission and, as you would expect, it was all neat and shiny with plenty of blue lights - except when I asked what the rather dirty video cable hanging from the ceiling was for. I was told it was the main (and at that time only) HD feed coming in from Germany. I wished I hadn't asked.

So, other than the "odd" bit of routing, all seemed well three days before the World Cup: the channel had launched and the promo loop was playing and getting a big audience! I had even managed to get away early on a sunny afternoon, then I had a call from an old editing colleague - "did you know your channel was out of sync and it looks like it looks like it's getting worse?".

Okay - don't panic - just call Red Bee to see what's happening.

They checked and told me that all seemed fine but that they would keep an eye. I trusted my source and needed to know more and as my Sky HD box had arrived the day before and I spent the evening watching the promo as it cycled and yes, by about 10pm it was defiantly out of sync.

The next morning it was still out when I checked just before I left for work. The previous night, I had e-mailed Seetha to say that we may have a problem - I could just imagine her reaction when Gary Lineker started to speak at the opening of the World Cup and a second or so later his mouth moved.

The more we checked the more confusing it got. The promo was definitely in sync on the original tape and on the server original server file and there was no point in the chain that could put the signal out of sync. We needed to see some live pictures through the chain and they needed to see sync on and we needed to do it quickly. So for all those people who contacted the BBC audience helpline (and me), the reason the London news went out on the HD channel in two short bursts one afternoon was so that three guys at Kingswood Warren could stare at the newsreader's mouth!

To the relief of everyone, the news was in sync - so the footie was safe.

But that didn't explain why the promo was going out over a period of time, would it do the same when we started transmitting real programmes from the server? Mike Taylor from Red Bee did some sterling work and tracked it to an "oddity" in the playout software. It's quite unusual to play out a programme again and again (and again and again) from a broadcast server and after a few days a very small error had built up, enough to see. Simple to fix once you knew what it was - but for 24 hours it wasn't an experience I want to have again!

Two days later, the World Cup kicked off and then the "fun" really started...

Andy Quested is Principal Technologist, BBC Future Media & Technology.