Comments for http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html en-gb 30 Sun 05 Jul 2015 04:36:55 GMT+1 A feed of user comments from the page found at http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html jonpojonpo http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=94#comment16 This data really Isn't difficult to interpret AT ALL. and despite the full files running to several gigabytes the actual information contained within them runs to only a few tens of megabytes which is well accesable to anyone with a handle on excel. which is very high level and easy to understand. While the Guardian has done a reasonable job. I decided to setup a website to allow people to dowload and explore this data in a better format at www.publicspendingdata.co.uk anyone should be able to analyse the easy to download CSV/html files. no sign up needed. no ads. just data. Thu 10 Jun 2010 08:51:15 GMT+1 LanceM http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=88#comment15 Rosslyn Analytics, a London-based technology company that specializes in enabling organisations to quickly and easily obtain spend visibility, has launched a dedicated portal that gives the general public the ability to view the UK government’s recently published public sector data from COINS. This portal can be found at https://rapidgateway.rapidintel.com. Mon 07 Jun 2010 14:55:24 GMT+1 Menedemus http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=82#comment14 Talking about data overload. It seems the former Labour administration were guilty of gross obfuscation as well as a desire to prevent freedom of information requests.As John Redwood has pointed out in his Blog, the Red Book used to be 47 pages deep detailing the profit and loss of government spending within Whitehall Departments. Under Labour the Red Book has grown to become in excess of 2000 pages with much party political point scoring embedded within it.Hopefully George Osbourne and the Coalition will also decrease this assault upon the senses and achieve declaration of reality rather than excuses and coverup for overspend or wasteful spending by the government of the day. Sun 06 Jun 2010 22:38:08 GMT+1 Paul http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=76#comment13 I can't download most of that stuff on the website because it is Bit Torrent... as one Government official said before the DEA "torrent is clearly only used to obtain illegal material".Yet now they see the advantage of Bit Torrent. Hmmm.Yes, sorry this was off-topic. Sun 06 Jun 2010 21:58:26 GMT+1 Steztric http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=70#comment12 @SadButMadLad: It is most likely a legacy system. The UK government have tried twice now to modernise the NHS computer system and failed both times, wasting millions of pounds.The problem is that there are probably hundreds of computers spread out througout the country, probably with specialised software accessing this database. It is not that easy to just change stuff :P Sun 06 Jun 2010 20:48:56 GMT+1 Barbazenzero http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=64#comment11 I have successfully downloaded the PDF "crib sheet"from the main COINS as well as the four zipped data TXT files from the COINS data page, using the "regular" zipped links rather than the BitTorrent ones.Two early points noted from process which are not clearly covered in the PDF are:● The two larger "fact" tables are too large to be handled by FilZip [an excellent  freeware zipper & unzipper] but can be extracted with the free evaluation version  of WinZip.● The text files themselves are in Unicode, which makes them twice the size they   probably need to be, since I find it hard to believe that HM Treasury have needed  non-Latin character sets since the asian colonies were given their independence.All the files can be opened - eventually - in Notepad or Word but I have not been patient enough to see them open in MSIE8. Knowing they're in Unicode should make life much easier if wanting to import them into a database. Sat 05 Jun 2010 11:02:52 GMT+1 BluesBerry http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=58#comment10 Oh my goodness! Talk about complexity!This morning, the Treasury has released a trench of data, and since this morning, that's where I've been sitting: in my trench looking up at this mountain of data.I can see why the Labour government turned down freedom-of-information requests from everyone & anyone because each request would generate so much confusion and subsequent inquiries. The only good thing is that George Osborne promised to release it, and George Osborne has released it. As for transparency, is there somewhere that I could take a course, a ladder to climb out of my trench? There is no public transparency here. I know this; the government knows this. So who are we kidding? Without training, COINS is not transparent to the lay person.I agree with Labour's points:1. 23 million lines of raw and unvalidated data and a high volume of follow-up enquiries would be too disruptive to the Treasury's work. Yep, I can see that.2. Another factor Labour quoted was the "impenetrability of the information to a lay user". Yep, I can also see that.As far as current data being released, it would be subsequent year's data before I found what I was looking for anyway.In addition to "Open Knbowledge Foundation", I also find this information site useful:http://www.info4local.gov.uk/ Fri 04 Jun 2010 18:45:53 GMT+1 BigRog http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=52#comment9 The reason that they will limit the data is three fold.Some spending is allocated and not yet spent, despite several warnings certain departments do not "give up" money that has yet to be spent and carry this over to the following year and only tell treasury after the fact.Two agreed spending is not always shown in the year it was actually spentThree certain monies are "hidden" by the current Government to pay for things that the public is not told about, but are phrased as being in the public interest. Treasury has a "slush fund" as do many Departments and this money is usually labelled up misc expenses Fri 04 Jun 2010 13:41:56 GMT+1 batzben http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=47#comment8 Whilst it may be impenetrable this isn't strictly speaking raw data. The Departments and other bodies all have there own accounting systems which are mapped to a centrally defined chart of accounts and uploaded into COINs. It's all part of the dream of producing Whole of Government accounts for UK plc.Has anyone found lines for Lloyds, RBS, HBOS etc. in there yet? Fri 04 Jun 2010 13:10:03 GMT+1 Ian http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=41#comment7 @U-ManI've been able to get the zipped files with no problem. I've not tried the BitTorrent links though. Fri 04 Jun 2010 12:11:53 GMT+1 U-Man http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=35#comment6 I have to say it is somewhat dismaying that as soon as I click on the treasury's links, they fail to open or inform me that "Windows cannot open" the type of file I am dealing with. I heard that these file types deviated from the norm, but I am very shocked to discover that they are wholly inaccessible...Am I missing something or does anybody else have this problem? Fri 04 Jun 2010 11:48:56 GMT+1 KDMINX http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=29#comment5 SadButMadLad wrote: >Why does the Treasury use the very uncommon terms of MiB and GiB >for the file sizes? Just about no-one uses these terms. Everyone >uses the common and defacto terms of MB and GB.>>For info: 1GiB ≈ 1.074GBMiB and MB are the same thing. MiB is prefered because "Mega" means different things for data and other things thus it conflicts with SI units.EG:1 MB = 1024 kilobytes1 Mm = 1000 Kilometers Fri 04 Jun 2010 11:34:01 GMT+1 piggeh http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=23#comment4 Why don't you campaign for greater contract information as no one ever seems to know the details for 80%+ of the contracts they have (take a look at the various public contract registers and you'll find it fairly sparsely populated despite their obligations to make such information public!). Fri 04 Jun 2010 11:26:35 GMT+1 Edwin Cheddarfingers http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=17#comment3 @ #2 SadButMadLadMiB and GiB etc. are designed to refer to powers of 1024 (or 2 to powers of multiples of 10), which is what historically a kilobyte and gigabyte have referred to, however this prefix is ambiguous because in SI units mega and giga refer to powers of 1000, so a megabyte can mean either 1,000,000 bytes or 1,048,576 bytes.So the likes of MiB and GiB have been introduced to remove the ambiguity, 1 MiB unambiguously means 1,048,576 bytes, 1 GiB unambiguously means 1,073,741,824 bytes which gives you your 1.074, however your assertion that 1GiB = 1.074GB is not necessarily true for the afformentioned reason that in some contexts 1GB may in fact be 1,073,741,824, in others it may be 1,000,000,000.As a computer scientist for some time, I don't really like the new terms either if I'm honest, I've never met anyone who ever uses the SI prefixes to mean powers of 1000 rather than powers of 1024 at least without explicitly saying so beforehand. The new terms seem to cause more confusion than the ambiguity of the old terms ever has! Fri 04 Jun 2010 11:26:11 GMT+1 RedGreenInBlue http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=11#comment2 "[Labour ministers] argued [1.64Mb PDF] that potential misinterpretation of 23 million lines of raw and unvalidated data and a high volume of follow-up enquiries would be too disruptive to the Treasury's work."If this argument abour the "impenetrability of the information to a lay user" and consequent "misinterpretations" of data is taken to its logical conclusion, findings of scientific research should not generally be available to MPs because most MPs are not scientists and therefore "lay users" who might misinterpret the data. I hope that even those Labour ministers would be able to spot the flaws in their argument.But of course, that's not the point. It's information about how public money is spent, itself collected using public funds, and therefore the public is entitled to see it. End of argument. Governments could easily avoid misinterpretations of the data by ensuring that their own analyses of the data stand up to independent scrutiny. If they don't, then any disruption is fully deserved. And surely politicians, more than anyone, should realise that withholding of data always looks suspicious. (They should have been especially aware of this after the recent UEA e-mail theft, where just the perception that climate scientists had attempted to withhold raw data caused serious damage to their public reputation, even though the vast majority of the data was already freely available online, and the unreleased data was mostly not theirs to release because it was licensed from third-parties such as national meteorological organisations which wanted to make some profit out of that data.)Oh well, it's a bit academic now, since Labour is no longer in government. Let's hope that this release of data is not a one-off, since the Conservatives themselves didn't seem to keen on FoI when they were last in government! Fri 04 Jun 2010 11:03:58 GMT+1 SadButMadLad http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=5#comment1 Why does the Treasury use the very uncommon terms of MiB and GiB for the file sizes? Just about no-one uses these terms. Everyone uses the common and defacto terms of MB and GB.For info: 1GiB ≈ 1.074GB Fri 04 Jun 2010 10:57:38 GMT+1 Opaque http://deltawww.live.bbc.co.uk/blogs/opensecrets/2010/06/coins_data_now_published_pleas.html?page=0#comment0 Just because it's massive and difficult is no reason for it not to be published so nice to see that the promise has been met :0What I find interesting is the fact that they are being clever enough to use torrent files to share this data. Fri 04 Jun 2010 10:37:01 GMT+1