FOI: Some end-of-year reflections
I think we're at an intriguing stage in the development of freedom of information in the UK. A year ago the context surrounding FOI was dominated by the role it had played in the MPs' expenses saga and the repercussions. Now that we're moving on from that particular while important matter, we can discern some interesting general themes - so here are some of my end-of-year reflections.
Data, data, data. That's what we've been getting - and that's what we're going to get more of.
This year has seen a rapid increase in the flow of government data being released to the public. It started during the previous Labour government and has accelerated under the coalition, particularly so far with financial data. And there's plenty more to come.
For the coalition this is clearly partly driven by the notion that transparency cuts waste and saves money (a point which the Information Commissioner Chris Graham is also keen to make when he gets the chance). But there's more to it than that - partly due to the work of some devoted campaigners, the zeitgeist is now for opening up government data and giving outsiders the chance to work with it.
For me, the change in attitudes is encapsulated in one small example. You can now see the location of London Underground trains in real-time here. The private individuals who created this map exploit data made available by Transport for London. In contrast a few years ago London Underground refused to send me a District Line timetable.
As a journalist, one of the consequences I reflect on most often is naturally how the data deluge will affect journalism. One of my colleagues, Mark Easton, wrote about some of the difficulties when large quantities of spending data were released last month.
Data journalism is still very much in its infancy. But it's bound to develop and become increasingly important. And I think we will see journalists competing for what I call "scoops of analysis" - getting the story first in data journalism will mean doing the calculations and finding the insights in the data which no one else has yet spotted and analysed.
The phrase "scoops of analysis" was actually used, as I recall, previously in the 1980s and 1990s ("popularised" would be rather too strong) by the former BBC Director General John Birt, although at the time of my writing this Googling it surprisingly produces no results. The Birtist doctrine was that journalism should have a "mission to explain", and he liked the analytical approach. This view was far from universally popular within BBC News, where many staff preferred to think of journalism as story-getting.
But the context is now very different. I envisage that from now on breaking news (and that means story-getting) in the field of data journalism will increasingly be based on scoops of analysis.
More generally than the issue of data release, the impact of the freedom of information system is both about policy and delivery.
We discovered this year that the prime minister who introduced FOI thinks himself a "nincompoop" for so doing. Tony Blair now worries about its effect on frank exchange of information within government. This is much talked about anecdotally, although an important academic study issued this year did not find FOI had such a chilling effect.
Tony Blair with his former chief of staff Jonathan Powell
I was particularly intrigued by the mixed messages in the discussion of this topic in The New Machiavelli, the (incidentally very entertaining) book published this year by Jonathan Powell, Mr Blair's former chief of staff.
He says FOI has resulted in a "reduction in the amount of confidential work conducted on paper", but he also writes "I have often been told that I should simply stop sending emails, but you cannot do that if you are running a place like Number 10." There is a practical conflict between caution and effectiveness, a tension which will doubtless remain.
But whatever the merits of the policy itself, for its first few years delivery has been a problem. Until recently, if you'd asked me for one word to characterise the UK's freedom of information system, I would have answered "delay, delay, delay." Now I might merely say "delay".
In his first 18 months in office the Information Commissioner Chris Graham has managed to speed up the rate at which his office assesses complaints. One measure of this is that, according to the latest caseload report, his office now has no complaint under consideration for over two years. The fact that this statement counts as good news indicates how bad the situation had become. At the start of this year [180KB PDF] there were over 30 such cases.
This is a crucially important change to how FOI works in practice. My impression anecdotally is that the ICO's progress in tackling the backlog it was struggling with is having a knock-on effect on the efficiency of the rest of the FOI system - it has made public authorities somewhat less complacent about their own delays. And Mr Graham now clearly feels more confident about putting pressure on authorities with particularly bad records to improve [152KB PDF].
I expect that, for many, the current controversy they most associate with the term freedom of information is that of Wikileaks and the US diplomatic cables - although, strictly speaking, as a leak it's an entirely separate matter from FOI. But one thing this does illustrate (as indeed did the MPs' expenses controversy) is that leaks can go much further than material the authorities officially if reluctantly decide to release in reply to FOI requests. Although the BBC has obtained under FOI some rather interesting reports sent back home by British ambassadors, we have been denied other more sensitive ones.
Much has been written about Wikileaks and how much the public should know about diplomatic communications, but there is one point I think has been largely missing - and it connects to large releases of data both under FOI and of the government's own volition.
I've heard civil servants say that the public disclosure of spending data and its reporting in the media has actually given them much easier access to information about state expenditure, in some cases to material which they felt was useful to their job and previously hard to obtain.
Similarly, of the three million or so Americans who apparently had access to Siprnet, the network which contained the cables leaked to Wikileaks, how many I wonder had actually become aware that (to take for example the first major revelation) Arab leaders had been pressing the US to attack Iran?
Widespread revelations of state information may or may not be in the public interest; but they're not only to do with what the citizen knows about the state, they are also to do with what the state knows about itself (and yes, sometimes some of us in the BBC learn about other parts of the BBC due to the BBC's own FOI disclosures).
Another current point of interest is the position of private companies in numerous different fields in relation to information access. The coalition government has extended FOI to academy schools in England (in legal terms they are private not state institutions). The Scottish Information Commissioner Kevin Dunion has been pressing hard for FOI to be extended in Scotland to various private bodies such as private prisons and Glasgow Housing Association.
But a recent Upper Tribunal decision [140KB PDF] ruled that the private water companies of England and Wales did not have to abide by the Environmental Information Regulations. I suspect we may see further developments in this area.
And finally, we're waiting for the coalition government to announce its plans for developing FOI.
It promised [476KB PDF] that it will extend the scope of the Freedom of Information Act but we have yet to see the detail of what it will do. However ministers have also talked of ensuring the process is not "swamped" by "people who overload the system". They say they're not sure the balance is right at the moment.
We now know where Tony Blair felt the balance should lie on freedom of information. We may soon find out more about the thinking of the current government.