« Previous | Main | Next »

Andy Finney: Was modern internet technology influenced by Domesday?

Neil Copeman | 12:13 UK time, Wednesday, 25 May 2011

 

Article written by Andy Finney

Andy Finney

Andy Finney

Producer on the original BBC Domesday Project

The discussion pages on Wikipedia are sometimes more interesting than the articles.

The BBC Domesday Project has its page and, until recently, it included a note about the kinds of online applications we know and love, such as mapping and Google Street View, that had their equivalents in the Domesday Project.

Any mention of the BBC being a red rag to certain Wikibulls, this note was expunged by an anonymous editor who was concerned about "weasel words and no citations to any reference that this project in any way 'led the way' or 'inspired' any present mapping software or service."

This got me thinking. What key 'new' bits of technology were there in, and were we first, and is it possible that the likes of Google were influenced by this? My colleagues will probably come up with a few more (and that's what comments are for) but here's my set.

Mapping

Some Domesday features just grew organically. Once we knew that the UK mapping had to be based on 4 by 3 blocks - the aspect ratio of 1980s TV screens - and there were existing paper maps that allowed us to show 4 by 3 km or 40 by 30 km in that space it seemed logical to allow horizontal and vertical jumps across those maps and to be able to 'zoom' from one scale to another.

It was a simple concept once an algorithm had been developed (my memory says by Dr Martin Porter) and worked by jumping between frames on the disc. Unlike current systems such as Google Maps and MultiMap it was not possible to dynamically render the maps on demand so paper maps for the whole of the UK had to be cut up, joined together where necessary and put beneath a video rostrum camera at the BBC Open University Production Centre in Milton Keynes.

The National Disc allowed users to select sets of data (such as population) and display these overlaid on maps on demand. This also was highly unusual in the mid 1980s, when computers managing statistics still usually occupied a whole air conditioned room and required white-coated acolytes.

Searching

Every photograph and every piece of text was allocated keywords based on a pre-determined vocabulary. This concept was well-established long before the 1980s (and today keywords sit in the invisible headers of every well-structured web page).

Where Domesday pushed the envelope a little was with the way the queries could be entered. We didn't feel that members of the public could (and should be asked to) use boolean logic with this and that but not the other. Dr Martin Porter came to the rescue.

Martin had produced a search system called Muscat early in the decade and had also developed what is now known as the Porter Stemming Algorithm. Stemming is the linguistic method by which your computer system knows that the words 'search', 'searching' and 'searched' actually refer to the same concept.

For Domesday this allowed the text searching to work out what the key words were when a user entered a sentence in the search box. In the original system you can watch as the system extracts the key words from your sentence and counts through the hits it finds on one or more of them.

Stemming in free text searching is used by today's web search engines. You can try it for yourself by putting in different versions of a word and seeing what the search engine returns.

The virtual machine

Whereas my original interactive video experiments had been written in BBC Basic, the Domesday Project required more sophisticated programming, notably to overcome the memory limitations of the BBC Micro and its 8 bit environment.

Fortunately Acorn had published a BBC Micro chip for a language called BCPL, which is often referred to as a forerunner of C, arguably the most influential computer language.

BCPL, which was developed In Cambridge by Dr Martin Richards, was a two-stage system whereby the program would be compiled into what was called CINTCODE (Compiled INTermediate CODE) and a second piece of software, called a virtual machine (VM) ran that CINTCODE on the computer.

In theory you could 'easily' change the VM and run the same code on another computer. Today the virtual machine concept is probably best known in Java. It should be possible to take Domesday's BCPL code and run it on a VM under Windows or Linux or whatever on a modern computer.

This process was used back in the 1980s for the Research Machines version of the Domesday System and to demonstrate Domesday running on an Acorn Archimedes computer.

There are four key reasons why the BBC couldn't just update the BCPL VM to run Domesday now: no source code (the BBC were not given it), translating disc commands such as frame numbers (for images) into file names, working around a few bits of the code that relied on the word size (number of bits) in the original system ... and the cost of doing the first three. Besides, even if this were achieved it would produce a stand-alone Domesday system, rather than the web-based one that the Reloaded team have made.

Virtual Reality

The Domesday team were aware of an interactive video project made around 1978-80 at the American University MIT which enabled a user to take a virtual trip around Aspen Colorado. It was called the Aspen Moviemap.

The MIT team used a specially-rigged car (not unlike the Google Street View car) to take thousands of still images of the streets and could then navigate around them from a touch-screen and an array of videodisc players.

We wondered whether we could use a simplified version of this idea in the UK and after some experiments decided we could use this to show the main types of housing in the UK and a few typical examples of 'the outside world'.

The Domesday walk model looks clunky compared to Street View and QuickTime VR since we took eight photographs at each point and the user can only turn in 45 degree increments, rather than smoothly. We also extended the idea to produce a way of accessing some of the contents of the National Disc, whereby the user could move around a computer-generated art gallery.

I had also hoped we might use the walk around a newly-built show house as a way of accessing information on sales of domestic products. Click on the TV and see the market for televisions. But we didn't have time to add that feature.

Crowd sourcing and Wikis

The internet provides an instant way of gathering information from people. Domesday aimed to do the same but using the Royal Mail as it's network. There were elements of a Wiki in the text pages of the Community Disc but, until now, there was no way for those pages to be rewritten and developed.

So it's a snapshot of a Wiki rather than a true Wiki but in the media landscape of the 1980s the notion of people writing whatever they wanted and it being published (and by the BBC at that) was very, very rare. And mentioning Wikis brings us back to where we started.

Like all innovative projects ... indeed like all projects of any kind ... Domesday was a mixture of existing ideas, ideas that developed from existing ideas, and new ideas. And it is important to remember that Domesday was a consumer product and was built with this in mind.

Could Domesday have influenced companies like Google? I did demonstrate it at America's main interactive video conference in 1988 and the project is often cited as part of what is known as the prior art when patents for interactive media are being considered in the USA.

Actually whether Domesday influenced 21st century interactive media is irrelevant. We tried out ideas that, in most cases, came around again and worked better because the technology was more suited to them.

Comments

  • Comment number 1.

    Each surrogate walk took a team of 4 photographers and a subject specialist who took on average between two and four thousand slides per walk. How useful digital cameras would have been had they been available back then.

  • Comment number 2.

    It is totally incorrect for the Wikipedia reviewer to say there was no "no citations to any reference that this project in any way 'led the way' or 'inspired' any present mapping software or service." A number of articles and papers were published in internationally reviewed journals e.g. see Rhind D, Armstrong P (the driver behind the Domesday project) and Openshaw S 'The Domesday machine: a nationwide Geographical Information System' published in the The Geographical Journal 154, 1, 1988, 56-68 (complete with colour pictures of the kit and sample output). I recall demonstrating the system in a major conference in 1987 in Boston where it aroused incredulity. I am clear that, whilst the technology of the laser disk in particular proved inadequate and marketing of the system by the BBC was almost non-existent, the demonstration of the feasibility of exploiting multiple data sets of many different kinds - including crowd-sourced information, government statistics (e.g. the population census) and maps - interrogating them via an interface that 5 year olds could cope with was pioneering and influential. Amazing what could be done in 128k!

    David Rhind

  • Comment number 3.

    @Going_Digital: The surrogate walk team was a little more streamlined than that. Each individual walk had a producer (me), a subject specialist, a production assistant and two photographers. One photographer shot the walk, using a Nikon F3 with a special back that held long rolls of film, and the other shot the details. The 35mm stills were optically printed (by Rank at Denham) into the 35mm movie format ... known in the trade as eight-to-four-perf ... to produce a roll of movie film that was then run through a Rank Cintel MK III telecine with computer grading (control of brightness and so on with frame-accuracy) at the BBC Enterprises facility in White City. The frame count in total for the walks was about ten thousand frames. I hadn't realised just how many there were till you prompted me to count them up. Every frame individually graded 'by hand' by the long-suffering telecine operator, Geoff Powell.

    @David Rhind: the Wikipedia editor was concerned that the Wikipedia entry had insufficient citations from current sources to back up the claims. The irony is that 'they' wouldn't allow anything on this blog to be used as a cited source because, being BBC, it would be seen as biased. So David, your task, should you decide to accept it, is to write a paper on the influence of Domesday for a current peer-reviewed journal. Then we can cite it.

  • Comment number 4.

    The information telling the computer how to link the thousands of pictures together into a walk was actually supplied twice on the disc, in two different files, which had some some slight differences between them.

    The first version was used if you reached the walk through a search. The other version was used if you entered it from the gallery. This version referenced a duplicate set of the 8 compass-direction photographs for the starting location. A door was superimposed on one of them to lead you back into the gallery.

    Other neat tricks were used so that you could walk into a lift on the ground floor of Trellick Tower, and in the time it took for you to turn around inside to face the door, the lift had already reached the floor where the flat was so you stepped out onto a different floor!

    If you turned left at the initial location you would read a brief description of the walk superimposed on the photograph. But when you moved away from the location and then returned, the text had disappeared - because you had actually returned to a replica of that start location. This meant there needed to be 4 copies of the 8 photographs for the start location of a walk.

    The walks varied considerably in size. The Brecon walk linked photographs at 302 locations, whereas the showhouse and terrace house each had just 26.

  • Comment number 5.

    Hi Andy,I am going to take some credit for the revival of Domesday.Earlier this year I contacted Philip Electrical,who worked in conjunction with Sony,And I was told they had no Knowledge of anything.I searched the B.B.C. they came back with nothing.My entry in Wiki was the only reference,having left a comment,within Wiki.You may have seen it.Cheers Colin Shorey

  • Comment number 6.

    Thanks for the informative article Andy, I didn't know that BCPL was a forerunner of C. Further evidence of Acorn being streets ahead. I trust you know, of course, that RISC OS computers are still going strong albeit in a much reduced market.

  • Comment number 7.

    Hello Andy. Are you sure the BBC were never 'given' the BCPL source code? Was this a condition of the original contract with Logica? Even if so, I'm sure it could have been released after enough time had elapsed for Logica to realise that it wasn't commercially valuable. I kept all of it for many years afterwards.

  • Comment number 8.

    Taffel, what was your role on the project, if I may ask? Do you remember much about the code after all these years? (If so, I'd be interested in having a chat with you.)

  • Comment number 9.

    Fascinating stuff - I had heard Martin Porter was involved but didn't know the details. I worked at Muscat with Martin in the late 90s and you may be interested to know that the Muscat search software was developed into an open source project, Xapian (www.xapian.org), which survived the disappearance of the original company. We're still using it in many customer projects - for example at the Financial Times. Martin's stemming work continues at another open source project, Snowball (http://snowball.tartarus.org/), which is used by many other search projects.

    Charlie Hull
    www.flax.co.uk

 

BBC iD

Sign in

BBC navigation

BBC © 2014 The BBC is not responsible for the content of external sites. Read more.

This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.