Public data: Free at last?
Go to a site called data.gov.uk this morning, and you'll find the fruits of a ten-year battle. It's being billed as a one-stop shop for developers hoping to find inventive new ways of using government data - and it was described to me by one man who's followed its long gestation closely as the "triumph of the geek".
Unlocking the huge quantities of data stored on computers across Whitehall and up and down the country in local government buildings has long been an obsession for a small group of dedicated enthusiasts. The issue brought together a combination of ambitious software developers who saw a commercial potential in all that data and cyber-utopians who believed that if the mission succeeded it would transform government and its relationship with citizens.
They made painfully slow progress, coming up against both incomprehension and inertia in Whitehall and some legitimate concerns about the cost of the whole project. Then in stepped a man who changed everything.
At a lunch at Chequers, the creator of the web, Sir Tim Berners-Lee, explained to the prime minister just why setting government data free was important, a vision he'd already outlined in a speech at the TED conference where he got the crowd chanting "raw data now!" I'm not sure whether he got Gordon Brown to join in with that chant around the Chequers dining-table, but he was given the job of making it happen.
In collaboration with Professor Nigel Shadbolt of Southampton University he set to work spreading the gospel through Whitehall - and surprise, surprise, when confronted with probably the nearest thing Britain has to a science superstar, the civil servants rushed to bring out their raw data and lay it at his feet.
I spoke to the two men about their efforts, Sir Tim joining us fittingly via a VOIP call after we somehow failed to fire up his ISDN connection, an older form of technology.
Nigel Shadbolt said Sir Tim had had a big effect on the civil servants: "There is star quality there; people are interested in meeting Tim and hearing the vision."
And Sir Tim said that while some people had concerns at the beginning - "will people accuse me of having 'dirty' data? Will they make unreasonable requests?" - they'd quickly realised there was some kudos in putting your data out there for people to use.
The site has been in beta for some months and now has nearly 3,000 datasets online, ranging from "Abandoned Vehicles (2003-04 to 2005-06)" to "Youth Cohort Study & Longitudinal Study of Young People in England". Already, some interesting applications have been developed - a school finder which lets you search local schools ranked by Oftsed score, FillThatHole, which uses ONS Census geography data to facilitate the reporting of potholes and other road hazards, and UK House Prices, a visualisation of property market trends using Land Registry data.
The mood amongst the "free data" community seems pretty upbeat about the whole project - one developer, Harry Metcalfe, told me that Tim Berners-Lee had really put the project on people's radars. He said that "low-hanging fruit" had been picked, but there was still a job to be done in liberating more real-time data, particularly from the transport sector. As we've seen by a couple of rows over iPhone apps, "public" data about things like train timetables and live departure boards is seen as a valuable private resource by its owners.
But can the Free Our Data crowd now be assured of victory? One man who's followed the whole project very closely is James Crabtree of Prospect magazine. He's optimistic but warns of some potential potholes - first, that when Tim Berners-Lee leaves, the spotlight will fade and it will just go back to being a geek issue; then, that despite the enthusiasm of both Gordon Brown and David Cameron, the politicians will turn to other matters. But Crabtree believes the key issue is all about maps, which are crucial to using just about any government dataset.
After a long-running row, it now looks as though the Ordnance Survey will allow free access to much of its mapping data. "But if that decision were to be reversed," says Crabtree, "then all of Tim Berners-Lee's work would come to nothing."
For now, though, prepare for a blizzard of new ways of manipulating public data, from crime maps showing which of your neighbours have been burgled to planning alerts telling you when the bloke across the road has submitted an application to build a two-storey extension. The data brigade says its mission is to use the government's own numbers to make all of our lives better. Let's see if it works.