Here is a ghost story for Christmas - it is a brief history of the appearance of ghosts and poltergeists and other spirits on television. Not fictional ghosts - but real ones, or the reports of their appearances, that you find in various news and documentary programmes.
But as so often when one looks at material in the archives, it turns out that it tells you less about the subjects of the programmes - the ghosts - than about the strange medium that possesses modern society - television.
In 1992 the BBC transmitted a drama that was based on a number of the factual reports I am going to show. The underlying aim of the makers of the drama was not just to frighten, but to demonstrate in a vivid way what had happened to the very idea of reality in television.
It was called Ghostwatch, and it caused a national sensation because thousands of viewers believed it was real. And, at the time, the BBC promised never to show it again.
I want to tell the story of the rise of the suburban poltergeist in factual TV from the 1970s onwards, how those reports inspired Ghostwatch, and how the extraordinary reaction on the night Ghostwatch was transmitted in 1992 showed clearly where the real ghosts of our society had now gone to live. They are inside television itself - a strange nether world of PR-driven half truths, synthetic personalities, and waves of apocalyptic fear.
In the 1950s and early 1960s the reporting of ghosts on television followed the classical rules. The hauntings were in old houses, stately homes, or ancient ruins. Here is a perfect example. It is from the Tonight programme in 1963. The reporter also follows an accepted format - he is indulgently sceptical, but brings with him a religious "expert" who is going to exorcise the presence.
But then, in the early 1970s, there was a peculiar change. The ghosts moved. They gave up haunting old castles and ruins and moved into the most ordinary suburban houses.
The battle between good and evil was now relocated into the suburban kitchens, bedrooms and even the stairs of modern Britain. Throughout, the ghosts also showed perfect taste in wallpaper.
Here is an extract from one of the earliest. It is the haunting of a council house in Swindon in 1973.
At this stage the film-makers are still following the classical editorial model. The local vicar brings in a religious "expert" to expel the poltergeist. The vicar smokes a fantastic pipe - and there is a wonderful shot of the Mr and Mrs Pellymounter watching the exorcism.
As the suburban hauntings multiplied in the mid 1970s, the approach of the programme-makers changed. The idea of exorcism disappeared and the TV reporters decided to turn to science. They would use special recording equipment to discover whether the hauntings were real, and the stories were turned into a battle between superstition and reason.
Here is part of a film made by the BBC Northeast regional magazine programme in 1975. It's about a 1960s block of maisonettes that have been built over an old disused coal mine just outside Newcastle.
The reporter and the crew decide to stay all night in an empty flat - and set up their special cameras and audio recording equipment.
There also a wonderful long-held shot in which one of the haunted occupants shows the reporter what the spirit did with his golf clubs.
The poltergeists kept spreading.
In January 1977 one turned up at 16 Ruskin Road, Dartford in Kent. Ann and Barry Robertson who lived there were terrified and are fleeing the house as the film starts.
There is a change in this film. The suburban couple at the heart of the story are no longer secondary figures in the story. They turn it into an emotional melodrama where they become the focus - Ann especially who has an epic turn of phrase:
"I can't even face taking the furniture with me because this thing - whatever it is - has interfered with my home. It's touched my things. And I'm so frightened that I won't even take the things with me now. So we're back to square one where we started. With nothing"
Suddenly suburbia becomes not boring - but sinister, mysterious and epic.
The film also interviews the man from Dartford Council who Ann and Barry are demanding rehouse them. He is sympathetic but then comes out with a great quote - "I'm afraid the Dartford Council Transfer Points Scheme doesn't recognise ghosts - and therefore they can't be pointed".
And then - ten months later in November 1977 - the Nationwide programme made a film which brought all the elements of the modern haunted house together into a perfect form. And it also introduced a powerful new character into the melodrama - the psychic investigator who was determined to prove that the haunting was real.
A poltergeist had apparently turned up in a house in the north London suburb of Ponders End in the borough of Enfield. The Nationwide film was going to make this house famous.
And along with the house, the film would also make a star out of this man - he was Maurice Grosse who was an investigator for the Society for Psychical Research. Maurice Grosse would come to dominate the TV-ghostworld interface.
The film is beautifully made. It is possibly the best evocation of the mood that is at the heart of all these film reports - a transformation of the dull interior of an ordinary suburban house into an intense psycho-drama where even the most mundane of objects, in this case a Lego-block, becomes possessed by an inner destructive force.
And the poltergeist has by now gone beyond wallpaper. It has chosen the most wonderful bedroom to live in. The walls are covered with Bay City Rollers and David Soul posters. And the shot of an elderly psychic investigator sitting among the images of late 70s teen dreams while listening to the recordings of himself communicating with the poltergeist is just brilliant.
The fascination with the Enfield haunting didn't stop there. Two years later BBC Scotland made another film inside the house.
This time they concentrated on the two daughters - Margaret and Janet Hodgson. The crew filmed the two girls as the poltergeist seems to speak through Janet, the strange voice coming and going in front of the camera.
It is weird and a bit frightening - but you also think that she may be faking it. And it is fascinating to watch the long held shots of the two daughters, studying their faces to try and work out what they are up to. And it introduces a new element into these haunting stories - that children are not innocent, but potentially malicious and a bit dangerous (like in The Innocents). A modern fear that was going to grow much bigger in the 1990s - especially again on TV.
The girls have since said that they faked some of the incidents in the house. But they insist that they were only doing this to test and tease Mr Grosse - and that much of it was real.
The Scottish crew had also got their own mini-scoop. They persuaded the police who had seen the chair levitate inside the house in 1977 to describe it. It is a fantastic two-shot.
The Enfield haunting became famous, and so did the psychic investigator, Maurice Grosse. He was completely convinced by the two Hodgson girls from Ponders End and it launched him on a thirty-year odyssey to try and fight against the rise of what he saw as a narrow-minded sceptical rationalism in Britain.
Grosse was a wonderful person. He died in 2006 aged 87. He had been trained as an engineer - and back in the 1940s he had become an inventor. His most famous invention was called "The Cost-Effective Poster Machine". It is better known as the rotating poster display which you can still see today at thousands of bus stops.
In 1976 Maurice Grosse's 22 year old daughter died in a traffic accident. It devastated both him and his wife. But then Maurice came to believe that his daughter was trying to make contact with him from beyond the grave. This led him to join the Society for Psychical Research - and that took him into the Enfield house just a year later.
Maurice Grosse was well aware that his quest to contact the supernatural was driven by the intense feelings of loss he had experienced through his daughter's death. This made him intensely sympathetic to the people he encountered in his investigations.
In 1996 Grosse made a Video Diary with the BBC. He went around with a Hi-8 camera, operating it himself. He then had full editorial control - and used it to put together a beautiful and moving film.
It is structured around various of his visits to hauntings - both past and present - but he uses that structure to also tell the story of his life - both factual and emotional. He describes his daughter's death and the feelings that raised in him, and the odyssey it led him into, in a very moving way.
One of the most touching moments is when he sits in an ordinary living room and talks to a couple who believe their dead son appears to them on their television. Grosse himself then becomes overtaken by emotion and has to leave the room
As you watch the film it becomes clear that Grosse believes that it is these intense feelings that give people, and the places they live in, the power to summon up poltergeists. The feelings give people something special - the power to pierce through the disappointing reality of their suburban lives and enter into something new and special. Another, and possibly better, world of high drama and raised emotion.
In 1988 a TV dramatist called Stephen Volk had an idea for a six-part drama based on all these suburban hauntings - the story would focus on how television had reported them. Volk's original idea was to have a TV reporter team up with a psychical researcher to investigate the haunting of a contemporary London council house. It was going to culminate in the final episode with a live broadcast from the house - and all hell was going to break loose.
Then Volk's producer, Ruth Baumgarten, suggested that instead they make a one-off play based on the sixth episode. Volk agreed. And he immediately realised that he could use the structure of a live outside broadcast to make a powerful drama that demonstrated dramatically what was happening to television as a medium - how the line between reality and fiction was getting blurred.
Out of that came Ghostwatch.
A few years ago Stephen Volk wrote a fantastic essay about the making of Ghostwatch. It was published by the Fortean Times. And you can find the whole thing here.
In it Volk describes his underlying aim - to make people look at what was happening to reality on television:
Ghostwatch was, of course, also about television.
It's quite difficult now to think back to the televisual landscape of 1992. Formats that dissolve the boundaries between factual and fictional TV have since become the staple diet of the schedules, and it's difficult to imagine a world where they were new or unusual. But this was the time of the first successful hybrids: docu-dramas and drama-docs. Drama series like NYPD Blue increasingly employed a hand-held camera style derived from documentary realism, and documentaries like Crimewatch and 999 were full of reconstructions using actors mix-and-matched to real footage of real people.
Ruth, the producer, and I discussed how we both felt we could no longer trust what we were seeing, what we were being shown or told by TV. The lines between the once distinct languages of factual and fictional TV were becoming dangerously blurred. Even the CNN Gulf War reports on Newsnight (with the infrared camerawork we duplicated in Ghostwatch) felt suspect, somehow unreliable. What was drama and what was not?
But then Volk added a line that I think goes to the heart of what has happened to TV ever since. The strange paradox that, at the very time that the audience is becoming more and more aware that not everything on TV is real, that same audience feel that if an event appears on TV - that is a guide to whether it is real or not.
Yet, paradoxically, television had also become the arbiter of reality, as John Waite exemplified on hearing of the release of his hostage cousin Terry in November 1991: "I won't believe it until I see it on TV."
Ghostwatch was transmitted on Halloween 1992. It was quite obvious from both the introduction and the titles that it was a work of fiction. But the reaction was astonishing - thousands of people rang in - either terrified or angry or to report that they were experiencing paranormal activity in their house at that very moment.
The next day there was a media storm - and the BBC reacted in its normal courageous way by burying the programme and disowning it. The Radio Times was apparently told never to mention it ever again. And Volk has described how it was like being airbrushed out of a photograph in Stalinist Russia.
But the extraordinary reaction rather proved the central aim of the drama.
It demonstrated the truth about modern television - that we all know that increasingly the line between fiction and non-fiction is blurred on TV. But far from making us distrust television this actually makes it more powerful. It possesses our imagination more powerfully precisely because we don't know what is real and what is not.
I think the reason is that, from the early 1990s onwards, the big confident stories of our time started to collapse, and people were faced instead with an everyday reality composed only of small and mostly mundane fragments. In the face of that, factual television has increasingly become a two-dimensional version of our world where everything is amplified and distorted.
News reporting and factual television are populated today by a strange nether world of PR-driven half truths, synthetic personalities and waves of apocalyptic fear. It is a world that is like ours but is exaggerated - weird, wonderful and frightening.
It is just like living in a haunted suburban house on the fringes of North London - except that it is now the whole world. All the mundane and banal aspects of reality are taken and infused with an hysterical intensity - that we are both fascinated by and terrified of - whether it be food or Al Qaida. Yet we know in our hearts that much of this is either distorted or just untrue.
It is the true spirit world of our time
It is made even weirder because, at the same time, audiences are shown harsh and terrifying moments of reality, but they are also insubstantial 2D images flickering on a box in the living room. They don't feel real, they look like a ghost world. Here is an example.
The tiny white figures you see that look like ghosts are actually still alive. But probably not for very long.
And here are some extracts from Ghostwatch - which show how much it was rooted in the suburban poltergeist reports of the 1970s. But also how it used them to brilliantly evoke the mood at the heart of today's television - where so much is half-fiction and half-real.
It is also very frightening - and a brilliant piece of TV drama. Just remember it's not real.