Section 10: Politics, Public Policy and Polls
Opinion Polls, Surveys and Votes
When we report the results of any research, especially when information is being summarised, the audience must be able to trust that the journalism behind what they see and hear is robust, the research is reliable and meaningful and the language used is both consistent and truthful. This accuracy, clarity and credibility is as important when we report on "polls" and "surveys" as it is in the rest of our journalism.
When we commission "polls" or "surveys" ourselves and invest them with the BBC's authority, we must take even more care to ensure we have made good judgements about their relative importance and the audience can trust what we are saying.
Commissioning Opinion Polls
When we commission opinion polls ourselves and disseminate such research in the name of the BBC, the methodology and the data, as well as the accuracy of the language, must stand up to the most searching public scrutiny.
We must take care to ensure that a poll we commission is not used to suggest a BBC view on a particular policy or issue. It is particularly important that a BBC poll is not used to imply BBC intervention in a current controversy.
The BBC rarely commissions polls on voting intention or other indications of party political support. Any proposal to commission an opinion poll purporting to sample party political support or voting intentions must be referred in advance to Chief Adviser Politics for approval.
We should normally avoid running joint polls with other organisations as they often carry particular problems of impartiality in their presentation.
We should take particular care in commissioning opinion polls seeking the views of children and young people. Advice should always be sought from Chief Adviser Politics.
There is a particular risk to the perception of our impartiality if a poll is commissioned by the BBC and carried out but not used, especially on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area. It can lead to claims that the results failed to match a perceived BBC view or intended narrative. Before such a decision, Chief Adviser Politics should be consulted.
Any proposal to commission an opinion poll on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area, for any BBC service, must be referred to Chief Adviser Politics. Technical advice, for example, on question design, is available from the Political Research Unit.
Polling is generally conducted face to face or over the telephone. Some companies conduct polling over the internet. As there is some debate about the reliability of internet polling, any proposal to commission an internet poll on any subject must be referred to Chief Adviser Politics.
Reporting Opinion Polls
When reporting the findings of opinion polls (especially voting intention polls in the United Kingdom), whether commissioned by the BBC or others:
- We should not lead a news bulletin or programme simply with the results of an opinion poll
- We should not headline the results of an opinion poll unless it has prompted a story which itself deserves a headline and reference to the poll's findings is necessary to make sense of it
- We should normally report the findings of opinion polls in the context of trend and must always do so when reporting voting intention polls. The trend may consist of the results of all major polls over a period or may be limited to the change in a single pollster's findings. Poll results which defy trends without convincing explanation should be treated with particular care
- We should not use language which gives greater credibility to the polls than they deserve. For example, we can say polls "suggest" and "indicate", but never "prove" or "show"
- We should not normally rely on the interpretation given to a poll's results by the organisation or publication which carried it out or commissioned it
- We should report the organisation which carried out the poll and the organisation or publication which commissioned it, as well as the questions, results and sample size. This information too should always be shown in television and online graphics
- We should normally report the dates of the fieldwork, and include them in television and online graphics, and draw attention to events which may have had a significant effect on public opinion since it was done
- We should normally report whether the poll was carried out face to face, by telephone or over the internet
- We should report the expected margin of error in voting intention polls if the gap between the contenders is within the margin. Television and online graphics should always show the margin of error.
If we have doubts about the methodology or the bona fides of those carrying out the poll, for example companies which are new or based abroad, we should either reflect that scepticism appropriately in the way we report the results, or we should consider whether the data is sufficiently credible for inclusion in BBC output. If in doubt, advice is available from the Head of Political Research.
Even when an opinion poll has been commissioned in an appropriate way, we should take care not to use elements of the research inappropriately, for example by drawing conclusions from sub-sections of the respondents that are too small.
We should always bear in mind that even properly conducted opinion polls by trusted companies can be wrong. When we report the result of a poll, no matter how convincing it may seem or what the attitude of the rest of the media, we should always ask how much of the rest of our story, and its prominence, is dependent on the poll's accuracy. We should also bear in mind whether the level of scepticism employed in the language and the direction of our reporting would feel sufficiently detached from the research if its accuracy was subsequently called into question.
Where the results of an opinion poll appear out of line with expectations, or suggest something new or distinctive, we should take particular care to ensure that it does not receive undue weight in the absence of a reliable trend.
Opinion Polls at Election Times
Election Guidelines for each campaign, which will be agreed by the BBC Trust and issued by Chief Adviser Politics before the start of each election campaign, will include specific guidelines on the treatment of opinion polls during an election period.
No opinion poll may be broadcast on the day of the election until the polls close or, in the case of a European election, all the polls have closed across the European Union.
A survey, as against an opinion poll, is normally addressed to a smaller and specific group. This may be a group of individuals (such as constituency chairmen, MPs, and university vice-chancellors) or a group of organisations (such as health trusts, FTSE 100 companies and local authorities).
We must conduct surveys, such as those of specific numerically defined groups like MPs or health authorities, with care and must never report them as polls.
If audiences are told that a survey has been commissioned by the BBC, they must have confidence that it has a level of statistical credibility which justifies any claims or assumptions about how representative it is.
Any proposal to commission a BBC survey on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area, must be referred to the Head of Political Research and Chief Adviser Politics, who will need to give approval if the survey involves MPs, MSPs, AMs, MLAs or MEPs.
The survey must have:
- a defined and finite group whose opinions, policies or behaviours are being analysed
- numerical parameters agreed in advance, such as an acceptable minimum response rate
- an agreed methodology, including questions that are worded appropriately and posed consistently
- care taken with the language in reporting the results to ensure nothing is claimed which cannot be supported by the data
- clear guidance to other BBC outlets (including the press office) who may report the outcome, ensuring that adapting the language for other audiences does not alter the meaning or inflate the claims of the original research.
We must not mislead our audiences about the status of the information contained in a survey. When reporting the results, we should normally use actual numbers of respondents; percentages should only be used with caution and when contextualised.
There is a risk to the perception of our impartiality if a survey is commissioned by the BBC and carried out but not used, especially on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area. It can lead to claims that the results failed to match a perceived BBC view or intended narrative. Before such a decision, Chief Adviser Politics should be consulted.
We should exercise appropriate scepticism when reporting the results of surveys commissioned or carried out by other organisations and, where necessary, include a description of the methodology used. Care is required, particularly in news output, not to report such surveys in a way which leads our audience to believe they are more robust than is actually the case.
Focus Groups and Panels
We must not imply that the views of panels, however carefully selected, represent the views of the entire population, and they must not be used as a means of trying to estimate party support in the electorate at large.
Panels or focus groups, when properly selected, may be used to examine why certain views are held and not the extent to which they are held.
Any proposal to commission focus group research on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area, should be referred at an early stage to Chief Adviser Politics and the methodology checked with the Political Research Unit.
Phone, Text and Online Votes and Other Straw Polls
'Straw polls' - including phone, text and online votes - have no statistical or numerical value.
They can be an effective form of interaction with the audience, illustrating a debate, but they should only be used with an explicit reference making it clear to audiences that they are self-selecting and not representative or scientific. Such votes cannot normally be said even to represent the audience for the programme or website, they only represent those who chose to participate. This applies even when there is a large response.
They should not be referred to in our output as a "poll". The term "straw poll" itself is widely misunderstood and should normally be avoided in output.
Results can be given within the context of the programme concerned in terms of actual numbers or as percentages if it is appropriate to the size of the response. However:
- results should not feature in news bulletins
- we should not seek publicity for the results outside of the specific content area in which the vote was conducted. The summary of an online or text vote can be reported on the radio or TV programme, website or blog with which it is associated, but it should not normally be reported elsewhere in news, on other TV or radio programmes, on other BBC websites or in press releases
- when straw polls are carried out on the same subject at different times, the results must not be presented in a way which would indicate a trend
- straw polls should never be used to gather serious information on party political support.
We should be particularly careful about using straw polls on those controversial issues which are vulnerable to highly organised pressure groups. Their ability to influence the outcome, even when we make it clear such votes are not representative, has the potential to damage the BBC.
Any proposal to conduct a vote on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area, must be referred to Chief Adviser Politics. In the case of a website in a language other than English, the proposal must be referred to Chief Adviser Politics and the relevant World Service Head of Region or National Director.
Anyone proposing to carry out a phone, text or online vote must refer to the Interactivity Technical Advice and Contracts Unit (ITACU) and complete the appropriate approval process.
(See Guidance: Audience Interactivity, and Opinion Polls, Surveys, Questionnaires, Votes and Straw Polls)
We should always make it clear that vox pops only represent some aspects of an argument and do not give any indication of the weight or breadth of opinion. .
Vox pops on matters of public policy, political or industrial controversy, or on 'controversial subjects' in any other area, must be edited to accurately represent those whose opinions have been solicited and include an appropriate range of views.