I'm afraid I haven't been posting any stories recently. The reason is that I am in the midst of putting together a live show with Massive Attack. It's a joint production between the BBC and the Manchester Festival.

I've had quite a few requests to put up a film I made a while ago about Mrs Thatcher - called The Attic. It's about how she constructed a fake ghostly version of Britain's past, and then used it to maintain her power. But also how she became possessed and haunted by this vision.

I'm putting it up as a bit of a corrective to the terrifying wonk-fest that took over after Mrs Thatcher died. A conveyor belt of Think Tank pundits and allied operatives poured into the TV studios and together they built a fortress around Mrs Thatcher's memory that was rooted in theories about economics.

They did this because economics is the only language that wonks understand. It's a view of the world where they see the voters - the people who put Mrs Thatcher in power - as simplified consumption-driven robots.

What was missing was the fact that Mrs Thatcher was also a powerful romantic politician who created a strange but compelling story about Britain's past that connected with the imagination of millions of people. It was fake, but it was incredibly powerful because she believed it. And the power of her belief raised up ghostly dreams from Britain's past that still live in people's imaginations - long after she fell from power.

The problem with wonks is that they can't deal with emotion and feeling, and they don't like stories. It means that they cannot connect at all with the feelings and imaginations of the voters. Yet the think-tankers have built a sarcophagus of economic discourse around Westminster.

What we are waiting for is a politician to come along who can connect with our imaginations and inspire us about political ideas instead of boring us to tears.

The film stars a wide range of characters - including Flanagan and Allen:

Members of the Political Wing of the Irish National Liberation Army:

Deborah Kerr:

PC Claude Morrell:

Mrs Thatcher and her friend Airey Neave:

And the ghost of Winston Churchill:

Here's the film.

One of the guiding beliefs of our consuming age is that we are all free and independent individuals. That we can choose to do pretty much what we want, and if we can't then it's bad.

But at the same time, co-existing alongside this, there is a completely different, parallel universe where we all seem meekly to do what those in power tell us to do. Ever since the economic crisis in 2008, millions of people have accepted cuts in all sorts of things - from real wages and living standards to benefits and hospital care - without any real opposition.

The cuts may be right, or they may be stupid - but the astonishing thing is how no-one really challenges them.

I think that one of the reasons for this is because a lot of the power that shapes our lives today has become invisible - and so it is difficult to see how it really works and even more difficult to challenge it.

So much of the language that surrounds us - from things like economics, management theory and the algorithms built into computer systems - appears to be objective and neutral. But in fact it is loaded with powerful, and very debatable, political assumptions about how society should work, and what human beings are really like.

But it is very difficult to show this to people. Journalists, whose job is to pull back and tell dramatic stories that bring power into focus, find it impossible because things like economic theory are both incomprehensible and above all boring. The same is true of "management science". Mild-mannered men and women meet in glass-walled offices and decide the destinies of millions of people on the basis of "targets" and "measured outcomes".

Like economics it pretends to be neutral, but it isn't. Yet it's impossible to show this dramatically because nothing happens in those glass-walled offices except the click of a keystroke that brings up another powerpoint slide. It's boring - and it's impossible to turn it into stories that will grab peoples imaginations - yet hundreds of peoples' jobs may depend on what is written on that slide.

I want to do a series of posts that will go back and reveal the forgotten roots of some of this fake objectivity that surrounds us today. They will be a series of stories that show how over the past fifty years both the political Right and the Left have gnawed away at the idea of objective truth. Sometimes almost colluding together to help bring about today's uncertainty and confusion about where power and influence really lies in our society.

The first is an odd story - with a very strange character at its heart. It is about how in the 1950s the richest man in the world, an oil billionaire in Texas, invented a new form of television journalism. It pretended to be objective and balanced but in fact it was hard core right-wing propaganda. It was way ahead of its time because, in its fake neutrality, it prefigured the rise of the ultraconservative right-wing media of the 1990s - like Fox News, with its copyrighted slogan, "Fair and Balanced"

The billionaire was called H. L. Hunt - Haroldson Lafayette Hunt. He made his fortune in the early 1930s by getting hold of one of the biggest oil fields in America - in the pine forests of East Texas. He was a ruthless, driven man and from early on he became absolutely convinced that he had superhuman qualities that made him different from other humans.

Here is a picture of Mr Hunt which gives you a sense of his conviction about himself.

From the 1920s onwards Hunt was a bigamist. He married two women and raised two families that were oblivious of each other. He told his second wife, Frania, that he was called Major Franklyn Hunt. There was a rocky moment when his picture was on the front page of all the Texas papers because of his spectacular oil deal. Frania asked Hunt if that was him - he told her no, that it was his uncle who had been so clever.

Hunt was part of a group of extreme right-wing oil men in Texas who had enormous influence because of their wealth. There is a brilliant book written about this group - The Big Rich by Bryan Burrough. Burrough describes how they had first risen up in the 1930s because they loathed President Roosevelt - "a nigger-loving communist", as one oil man called him. They were convinced that Roosevelt's New Deal was really run by Jews and communists - or "social vermin" as they politely put it.

A Texas congressman called Sam Rayburn summed up this group of right-wing oil men. "All they do is hate" - he said.

After the Second World War H L Hunt did two things. He added another, third, family to his bigamist's collection. And he also turned to the new medium of television to promote his ultraconservative views. In 1950 he wrote a pamphlet putting forward the idea of what he called an "Educational Facts League" - its purpose, Hunt wrote:

"will be to secure a impartial presentation of all the news through all the news channels concerning issues of public interest"

It would, said Hunt, be an organization where ordinary Americans would be supplied with the true facts of political life.

Hunt announced that the organization would be called "Facts Forum" - and he found a man called Dan Smoot to be its public face. Smoot had been an FBI agent - and he was smooth and reasonable. Starting on radio, but then moving to television, Smoot presented a show called Facts Forum which every week would give you, the audience, a balanced presentation of the facts behind the news. Very reminiscent of the later catch-phrase on Fox News - "We Report, You Decide".

In fact this declaration of balance and fairness was rubbish. Smoot would begin by presenting the left or liberal viewpoint on a subject in a dull, bland way. Then would enthusiastically put forward the alternative, or what Hunt called, the "constructive" view. This view was simple - all government was bad, business should be left alone - and anyone who disagreed was a communist trying to take over the world. And was probably a Jew as well.

The programmes were radically skewed to promote an ultraconservative agenda while pretending to be neutral and balanced.

There was lots of implied racism in the shows. In his book Bryan Burroughs quotes from one episode where Smoot argued against fair employment legislation - and said:

"Remember that the negroes when first brought to America by Yankee and English merchants were not free people reduced to slavery. They were merely transferred from a barbaric enslavement by their own people in Africa to a relatively benign enslavement in the Western Hemisphere."

Facts Forum became a successful media enterprise - with two syndicated radio shows and three TV shows produced from their own studios in New York. They were backed up by books and pamphlets paid for by Hunt. One was called "We Must Abolish the United Nations" - written by Joseph Kamp. His previous "balanced" books had included one called "Hitler Was a Liberal".

Here is a wonderful documentary profile of H. L. Hunt. It was made in 1968. By now his first wife had died, the second had got fed up and moved away, and Hunt was now left with only his third wife - Rita Ray.

You get a very good sense of Hunt's obsessive drive to promote his conservative views - sending out endless pamphlets, training young men and women to become part of his League of Youth Freedom Speakers, and even insisting that his whole family sit at the dinner table to listen to one of his new radio shows. It was called LIFELINE. Again Hunt was ahead of his time - because the show fused right-wing anti-communism with fundamentalist religion.

What you don't see is the tragedy of Hunt's life - his eldest son Hassie. He had originally followed his father into the oil business, but had then become violent and paranoid. Hunt had tried his own treatment - bringing in lots of women for Hassie to have sex with. But what had worked for the father didn't do much for the son. Doctors tried ECT - but that didn't work. In the end Hunt was persuaded to let them give Hassie a prefrontal lobotomy and his son spent the rest of his life wandering the Hunt estate like a strange ghost.

At the end of the film Hunt and his wife get up in their living room and sing together "We're just plain folks". It's very spooky. And it's not true.

Hunt's Facts Forum was the model for much of what was later to come with the rise of the right in the media in the 1990s - both in radio and TV. But Hunt didn't just shape the future of the right, he also had a profound effect on the way the Left too attacked and corroded the idea of objectivity and neutrality in journalism.

It happened because of some pieces of paper that were found in the jacket pocket of Jack Ruby - the man who shot Lee Harvey Oswald. Two of them were scripts from Hunt's radio programme called LIFELINE. The third had a telephone number of one of Hunt's sons.

Many of Lifeline's programmes had attacked John F. Kennedy as a communist dupe who was destroying America - and Jack Ruby had apparently been outraged by such vicious propaganda against Kennedy.

Then it was discovered that a full page advertisement placed in the Dallas Morning News on the day of the assassination had been partly paid for by another of Hunt's sons - Bunker Hunt. It was surrounded by a black, threatening border - and was titled sarcastically "Welcome Mr Kennedy to Dallas"

Like his father, Bunker Hunt was an ultraconservative - and the advertisement was placed under a title that echoed Facts Forum. It was called "The American Fact-Finding Committee" who described themselves as "An unaffiliated and non-partisan group of citizens who wish truth". And it accused JFK of all sorts of treasonous acts against America - including:

"Why have you ordered your brother Bobby, the Attorney General, to go soft on communists, fellow-travellers and ultra-leftists in America, while permitting him to persecute loyal Americans who criticize you, your administration, and your leadership?

We DEMAND answers to these questions, and we want them NOW."

As a result newspapers across America attacked Hunt's operations for creating the "climate of hate" in Texas that might have contributed to the President's death. And Hunt and his sons became targets in the FBI investigation that would then become part of the Warren Commission.

And it got worse. In 1967 the ambitious District Attorney in New Orleans, Jim Garrison, opened a new investigation into Kennedy's killing. Garrison started talking about how there had been a conspiracy that might have included certain unnamed Texas oilmen.

Hunt's head of security managed to get hold of a diagram drawn out by Garrison's team where "H L Hunt" was at the heart of a complicated network of lines drawing connections between the Dallas police, Ruby, Oswald, plus all kinds of small-time players in Dallas. And although Garrison's investigation folded in 1969 - it, and its diagrams, became the template for the growing conspiracy theories from the left.

One of the earliest - and most powerful - expressions of this was a film called Rush To Judgement made in 1967 by a left wing filmmaker called Emile de Antonio and a lawyer-turned-investigator called Mark Lane. De Antonio is a fascinating character - he came out of the avant-garde art world, and had worked with Andy Warhol and Robert Rauschenberg - and he shared their knowing distrust of the media world of two-dimensional images that was then becoming so prevalent.

Rush to Judgement sets out to propose an alternative explanation for Kennedy's assassination. At the heart of this other story is the idea that there is a group of powerful, shadowy men in Texas who used their wealth and power to create a distorted fiction - Oswald the lone nut - to disguise their conspiracy. A fiction that the public then believed.

The film interviews a whole host of extraordinary bit players from the Texas world and builds up a very powerful mood of uncertainty and suspicion. Underlying this is a message that says these hidden forces in America will never allow you to know the truth. Which means that what you are told by the media may be a lie. That you are being manipulated.

Just as H. L. Hunt himself was gnawing away at the idea of objectivity and truth through his own TV programmes, so too were the left also using a demonic caricature of H L Hunt to do the very same thing. He and other shadowy figures, the left said, will never let you know the truth.

Here is a section of the Rush To Judgement film. It had its world premiere in 1967 on BBC television - broadcast for an hour and a half at prime time. The section starts with the presenter in the studio introducing it - and framing how the viewer should interpret it. Then I have cut straight to the latter part of the film - which is all about how intertwined Jack Ruby was with the Dallas police and establishment.

It is long, but I have left it like that deliberately, because I think it is important to see how Emile de Antonio uses a particular technique to persuade you that he is presenting the real truth. The interviews are held long, and an archive interview with the Dallas police chief is used repeatedly to counterpoint them. It has a cumulative power that feels real and also feels like it is allowing you to judge the characters. That technique would rise up and become central to many of the more mainstream liberal documentaries of the last thirty years.

But it is also very much a technique borrowed from avant-garde cinema and in that sense is as artificial a language as anything you see on Fox News.

We report. You decide.

Mark Lane went on to help write a film in 1973 called Executive Action. It was about how a group of Texas oilmen kill President Kennedy. It was the same idea that resurfaced in Oliver Stone's JFK. But the best, and earliest, caricature of Hunt is in the film Billion Dollar Brain - also made in 1967. It was written by Len Deighton and directed by Ken Russell. The villain is a raving right-wing Texas oilman called General Midwinter who runs an organisation called Crusade For Freedom - modelled on Facts Forum and Lifeline - and wants to use his giant computer to bring down the Soviet Union.

Here's a short clip of General Midwinter in full-on Hunt mode.

But H L Hunt was far more than a caricature right-wing nutbag. The roots of so much of the distrust of the media today lie back with him and his ideas - with his Facts Forum in the 1950s and the strange role he played in Dallas in the 1960s.

In later posts I want to trace how what Hunt started, spread out from the dark pine forests of East Texas and began to develop into a much more powerful force undermining the idea of neutrality and objectivity in our age.

The West is worried about the rise of Islamism in Africa. There are two big fears - one is that there is a new international terror network that will come and attack Europe and America. The other is that sneaky Islamist groups like the Muslim Brotherhood will get themselves elected - and then promptly abolish democracy.

But behind these fears is an incredibly simplified - almost fictional - vision of the world. It possesses the minds of many western politicians, journalists and associated think tank "experts". And at its heart is a kind of filter that wipes away anything complex about power and the struggles for power in African countries - and replaces that with a simple picture of the world as divided between goodies (us in the west) and dangerous frightening baddies who are out to destroy us.

It's both blind and arrogant. And it's terribly dangerous.

To try and bring it into focus I want to go back twenty years and tell two dramatic stories. In them lie many of the roots of today's western fears - but also, in the details of both stories are keys to understanding two crucial things that we ignore today at our peril. One is the complex local power struggles that have helped the rise of Islamism in Africa, and the second is the way past western interventions have fuelled a hatred and distrust of Europe and America - that has in turn massively helped the Islamist cause.

One is the story of what happened in Somalia between 1990 and 1993 - the real events that led to Black Hawk Down or, to give it its proper name, "Operation Gothic Serpent". The second is the story of the weird and horrific events that happened in Algeria between 1992 and 1996 after the Islamist party called FIS was stopped from winning an election by an armed coup. A coup that had the implicit backing of the west.

There is an odd ghost that haunts not only Somalia's history, but has also lodged itself in the western imagination. He was called Mohammed Abdullah Hassan - and a hundred years ago he set out to try and unite all the Somali people in an Islamic state. The British called him The Mad Mullah and they battled against him for twenty years until they found a new way of getting rid of him. They bombed him from the air.

These are the forgotten ruins of the place that was going to be the capital of his Islamic state - he called it The Dervish state

For the next forty years the Somali people remained divided - ruled by the British and the Italians as part of their empires. Then, in 1960, Somalia was finally given its independence. But, like so many of the other former European colonies, all sorts of powerful remnants of colonial rule remained. Not just the arbitrary lines drawn on maps to make the new countries - but in the minds and imaginations of millions of newly liberated people.

Here is a film made in 1961 which captures this brilliantly. It's from a series called Africa Now, subtitled First Hand Reports from a Changing Continent and it is about life and politics and the new forces of power in independent Somalia.

The capital, Mogadishu, had been part of Italian Somaliland - and the film shows how strongly the Italian presence remains. Not just in the grand buildings that had been part of Mussolini's dream of a Second Roman Empire, but in the language. Not only is there no written Somali language - which means the Somalis use Italian - but they don't even have a word for "independence", so they use the Italian word - "indipendenza". I also really like the attempt to create a written Somali language. It was called "Osmania", and it wasn't a success.

The film also shows how Mogadishu has already become "the cockpit in the propaganda struggle in the Cold War". The film captures the ambassadors from all the different players - the Soviets, the Americans, the communist Chinese and the West Germans - going hither and thither in their gleaming cars in Mogadishu, all snuffling around trying to gain influence over the new President, Abdullah Osman Daar.

And the key to that influence is foreign aid. The film shows how the Soviets are offering to build a proper harbour, while fascinatingly the Chinese are already building a road system for Somalia. The Americans don't seem to be doing very well - but the West German ambassador is very keen, he spends his time walking around the desert looking for possible places for development projects.

But you can see who's going to win out. The Russian ambassador who is described as "a carefree agitator with boyish charm".

But in 1969 democracy in Somalia ended. There was a military coup led by Major-General Siad Barre who set up what he called The Somali Democratic Republic. But in reality it was a centralised communist state modelled on General Barre's interpretations of Marx and Lenin and Mao.

Siad Barre promised to wipe away the ghosts of the past that were holding Somalis back from being truly independent. And that meant not just the old colonial remains, but the crucial thing that was holding Somalia back, Barre said, was the clan structure.

Somali society was permeated by a complex clan structure. Somalis defined themselves and understood their relationship to each other in great part through this system of clans and sub-clans. Siad Barre said that it was the clans - or "clanism" - that had undermined democracy in the new Somalia - so he was going to wipe out this destructive and outmoded "tribalism" and replace it with a new, centralised society run by The Supreme Revolutionary Council.

In 1974 the Council published a book about the new society they were building. It has great images of revolutionary displays.

It also contained lovely colour pictures, like this one of modern Somalis dancing at the discotheque in the new Juba hotel in Mogadishu.

And it also summed up this glorious new revolutionary world and its beautiful future like this:

Algeria didn't get its independence quite as easily as Somalia. Between 1954 and 1962 revolutionary groups - the main one was called the FLN - fought a vicious terrorist war against the French who ruled Algeria. The FLN bombed French civilians in cafes and the streets, while they also killed many Algerians in the French controlled Algerian army. In response the French killed the guerrillas and also used widespread torture.

In 1962 the French gave up and Algeria became independent. Its first President was one of the leaders of the FLN - Ben Bella. But in 1965 he was deposed by a military coup led by one of his close friends from the revolutionary times - Houari Boumediene - who, of course, like Somalia, turned Algeria in the a copy of the Soviet Union.

It all worked fine for a while because Algeria had oil - and as oil prices rose the FLN used the money to subsidise their state socialism. But underneath everyone knew that power was really concentrated in a small elite group that came from the east of the country and excluded everyone else.

There was growing resentment, but no real coherent opposition. But then, in November 1982, there were a series of battles on the campus of the University of Algiers between a group of Marxist students and a group of Islamists who killed one of the Marxists. The Islamists were protesting about the fact that the Marxists, who all spoke French, would get all the best paid jobs. While anyone who just spoke Arabic would find it nearly impossible to get a professional career. This meant that they were excluded from power in Algerian society.

The protests were immediately repressed - the Islamists were all arrested. But it was an important moment because it was the first public demonstration by an Islamist opposition, breaking cover and coming into the open in a country where all opposition was banned. The protests were led by a teacher called Abbasi Madani who had once been in the FLN. He was put in prison for two years - but he will turn up later in this story playing a very important role.

They key thing in the protests was the fact that the Marxists spoke French - and that was the route to power. Again it was a powerful example of how the remnants of French colonial times still exercised a powerful grip on the destinies of those who were supposed to be free and independent of that past. And it was that frustration that was a powerful fuel for the growing Islamist movement.

The most dramatic example of how the French Empire still possessed the minds and behaviour of Africans was in the Central African Republic. It too had got independence from France in 1960 - but in 1965 there was, of course, a military coup and Colonel Jean Bedel Bokassa took power. In 1972 he made himself President for Life, but then, in 1977, he decided to crown himself Emperor of the Central African Empire.

It was a weird and grotesque demonstration of how the European mind set still controlled Africans in a distorted way. Because Bokassa was directly modelling himself on the French emperor Napoleon - and his coronation was supposed to be an exact copy of Napoleon's coronation as emperor in Paris in 1804.

Here is a great film made about Bokassa as he prepares for his coronation. It's a wonderful picture of what happens when a mad dictator decides to spend lots of money - clutches of European designers and planners and "facilitators" flock around all taking it very seriously. While Bokassa spends his time in his palace watching film of other royal coronations and the British Queen's Silver Jubilee in order to get inspiration.

Bokassa is also interviewed. He explains why he cuts peoples' ears off - he says it's a lot less barbarous than the death penalty, which France still had at that time. I suppose he has a point. And then he tries to explain why he is establishing an Empire when in fact he hasn't got an Empire. It's a very odd explanation - and it's very funny.

To the Islamists in Algeria, a figure like Bokassa was a dramatic example of what their fundamental theory predicted. Modern Islamist ideas said that European and Western ideas of democracy were always going to lead to corruption. However well-intentioned at the beginning, the system gave enormous power to individuals and that always corrupted them.

It was a very pessimistic theory because it saw human beings as always being fallible and corruptible. Bokassa was an extreme example, but the Islamists believed that the same thing had happened in Algeria. The idealistic Marxist revolutionaries had morphed into a corrupt and repressive clique. The only solution was to an impose a rigid, incorruptible system of moral and political guidance on the politicians which they had to follow. And that should be drawn from Islam.

The Algerian Islamists’ chance came in 1988. Two years before - in 1986 - oil prices had collapsed and the effect on Algeria had been catastrophic. Half of the country's budget was wiped out and the whole socialist "experiment" collapsed. Out of the disaster came widespread corruption and soaring prices.

On October the 4th 1988 the dam burst and the young, angry urban poor started to riot in Algiers. The centre of the rioting was in the shopping mall called Riad al Fath, the Victory Gardens. It symbolized the elite that ruled Algeria and the rioters smashed it up. In the next days the rioting spread spontaneously. And the only organisation ready and able to ride the wave of fury were the Islamists.

And on March 10th 1989 the Islamic Salvation Front FIS was formed - with the aim of bonding together the chaotic rebellion and using it to create an Islamist state. The founders of FIS had different approaches, but the two key ones were Abassi Madani and Ali Belhadj. Madani had led the protests back in 1982 and he believed that it would be possible to Islamize Algeria without changing the fabric of the state, while Belhadj was more radical - he believed in armed struggle to create a new kind of state.

Here are the first reports of the rioting in the "Days of October" - followed by the meeting that announced the founding of FIS.

Meanwhile in Somalia things were also going very badly for the "Victorious Leader" - Siad Barre.

His problems had begun back in 1977 when he had decided to try and create what he called Greater Somalia. Barre started by invading an area of neighbouring Ethiopia called the Ogaden. Millions of Somalis lived there - but back in the late 1940s the British, under US pressure, had decided it was part of Ethiopia. But now Siad Barre decided Somalia wanted it back.

To begin with the invasion went well. But then the Soviet Union, who had been backing Siad Barre, suddenly decided to switch sides and back Ethiopia. Almost overnight they pulled out their advisers and troops, along with a bunch of Cubans who had also been helping Somalia.

The reason the Soviets switched sides was because there was a new dictator running Ethiopia who was more Marxist-Leninist than Siad Barre - and even more ruthless. This made the Soviets feel that he was more worth backing and they poured weapons, money and men into Ethiopia to help defeat Siad Barre, their previous friend. This included airlifting thousands of Cuban troops into Ethiopia.

Here is Siad Barre saying everything is going swimmingly with the Russians - followed by news footage of the Soviet Advisers leaving Mogadishu a few days later.

And here is film of the weird, frightening world of Colonel Mengistu in Ethiopia that the Soviets went off to help. It was shot just after Mengistu had taken power in 1977. He had just started what he called The Red Terror. It was the mass execution of what Mengistu called "counterrevolutionary elements" - otherwise known as the Ethiopian Peoples' Revolutionary Party who had already been running what was called The White Terror, which involved killing Mengistu's supporters - who were members of what was called the Derg Party

There is a very odd scene in a graveyard where there are hundreds of graves already dug for the future victims of the White Terror. There is a very good piece of deadpan dialogue from one of the gravediggers.

Q. What is the Red Terror?

The Red Terror is conducted by members of the revolution.

Q. And who are the victims of the Red Terror?

Those who conduct the White Terror

The solution for Siad Barre was simple. He switched sides too - and went to the Americans for help. The US started to pour arms and money into Somalia. But it came too late to help him in the war in the Ogaden. The Russians and the thousands of Cuban troops mounted a counterattack and smashed the Somalian army. The Ethiopians then displayed the arms they had captured along with captions saying where the arms came from.

and their ammunition

I particularly like - "Reactionary Pakistani Grenades"

The defeat in the Ogaden was a disaster for Somalia. Over one million people fled from the Ogaden into Somalia, a country that then had a population of about 4 million. Food prices soared, groups began to fight for access to precious water, and there was total disillusion with Siad Barre.

Over the 1980s the Americans poured all kinds of weapons along with hundreds of thousands of dollars into the country. And as the economy collapsed the country became increasingly dependent on American aid. But the aid then had a strange consequence - it brought the clan structure back to prominence and power. Siad Barre gave up any idea of ridding the country of "clanism" and started using the aid as a way of buying loyalty from different clans. Clans who supported him got the aid, those who didn't went without. This had a crucial effect because those who headed the Somali clans began to see aid as the route to power in Somalia.

And when the Cold War ended in 1990 the clans who had been excluded set out to overthrow Siad Barre. A war began and Barre was forced out. It didn't stop there though, "the liberators" split into sub-clans and then started to fight each other viciously. The ensuing civil war had terrible consequences because the terrible violence was the primary cause of a famine in the Bay area of the country that surrounded Mogadishu. The victims were hundreds of thousands of people who had been displaced by the fighting.

What then happened was one of the great scandals of the past twenty years. The United Nations promised to help - and then did nothing. A bureaucracy that had once promised to bring peace to the world had become corroded and corrupted by the politics of the Cold war - and it failed utterly. For over a year it did nothing - and thousands of Somalis died of starvation.

Then western television discovered Somalia. Crews began to pour in from Britain and America and sent back horrific pictures of dying children. There has been much criticism of TV journalists both from within the aid community, and from those who think that aid is a bad thing - they argue that television simplifies, emotionalises and thus distorts the reality on the ground.

Here is a film that I think is very important in this debate. It was made by the BBC journalist, Michael Buerk, at the time in Somalia, and it is about the difficulties of reporting such a complex situation. It's good because it shows how chaotic and incomprehensible things were - with different factions in the civil war trying to get control of the aid (because that was what they had learnt from the 1980s onwards). But it also shows how television was inexorably drawn to the two powerful images that were beginning to occupy a very big space in the western imagination.

The innocent, dying child.

And the evil frightening men on their "technicals"

Goodies and baddies.

In Algeria, in contrast, things seemed to be going very well. The riots had forced the ruling FLN leadership to give up on the one-party state - and bring in proper democracy. In February 1989 President Chadli announced a new constitution that would allow political parties to exist and compete in elections on both a local and national level. It was an extraordinary breakthrough - and everyone had great hopes of a future democratic Algeria.

But there was a lurking doubt in many peoples minds. The front-runners in any future elections were obviously FIS - the Islamist Salvation Front - because they already had a national organization. But the question was - did they really believe in democracy? Or would they simply use the elections to get into power and then create a new kind of Islamist state that abolished democracy? No-one knew.

But FIS had the high ground because they offered an alternative to the corrupt regime. For the first six months of 1990 they organized marches and meetings - and then on June 12, 1990 FIS won an astonishing victory in the local and municipal elections. It won control of the majority of the country's communes.

Here is the first report on the BBC of the shock result - and already you can feel the concern in the west beginning. The fear that this would be Khomeini Mk II, though this time it was Sunni not Shia. There is also a good interview with a Tunisian Islamist who tries to counter these fears. He says that western governments must not turn away from this new Islamism:

"otherwise they will be seen to be supporting the corrupt governments, and our people will make the link between the dictators in our countries and British and American and European governments"

The mystery about FIS was personified in the two men who led the Islamist movement. One was the urbane Abassi Madani who drove around in a Mercedes and spent his time reassuring the Algerian middle class establishment that everything was going to be OK.

The other was the charismatic Ali Belhadj who rode around on a small motorcycle and was an amazing public speaker. His followers were the young urban poor. They were called the "hittistes" - from the arab word for a wall - "hit". They were called this because millions of hittistes spent all day leaning up against the wall with nothing else to do.

There was euphoria in FIS after the local elections - but the effect was to deepen the mystery and the fears. Groups of young FIS followers started to try and impose bits of what they thought were sharia law. Women who worked for the local communes were forced to wear the veil, video stores and shops that sold alcohol were closed down. And the middle class who had supported FIS began to get worried.

It got worse. At the end of 1990 Ali Belhadj gave an interview where he zeroed in on how western culture - above all French culture - had poisoned the very minds of Algerians. This had to be wiped out. His intention, he said, was:

"to ban France from Algeria intellectually and ideologically, and be done, once and for all, with those whom France has nursed with her poisoned milk"

The Hittistes loved this - and groups of them started to go round trying to destroy the TV satellite dishes that were feeding the poisoned milk into the minds of the Algerians.

The Algerian middle classes really didn't like this. They felt they were becoming trapped in a regressive bubble - isolated from the world that they connected with via the French news that came in through their dishes. They also feared the growing thuggishness of groups of Hittistes who were going round beating up girls who didn't wear the veil.

Sensing this disenchantment, the FLN rulers started to play dirty. They designed the new electoral constituencies deliberately so they would weaken FIS's chances of winning. In return FIS got very angry and in June 1991 they called for a General Strike. Thousands of their supporters took over squares in the centre of Algiers and things got very Arab Spring.

At a press conference Ali Belhadj made a dramatic intrusion. He said that if the ruling elite tried to stop FIS then he would take up arms and fight them, just like his father had done when he had fought the French. It's a really powerful moment that shows how charismatic Belhadj was - and also why the middle classes were becoming frightened of FIS.

In response the Algerian government declared a state of emergency, sent in the troops and postponed the elections till December. Plus they arrested both Belhadj and Madani - and threw them in jail.

Algeria was gripped by a dramatic crisis. Here is just one moment captured on video that shows that intense mood. Ali Belhadj's young son addresses a mass rally of Islamists. The son has an amazing power and charisma - just like his father. The response from the crowd also show just what the Algerian government were unleashing through trying to trick FIS out of their election victory everyone knew was coming.

Here too are parts of a really good report shot during the crisis in June and the occupation of the squares. It captures the mood of the time - and also shows the growing fear among the middle classes about FIS and the suspicion about what the Islamists were really up to.

But just as FIS were trying to force the poison of France - and the West - out of the Algerian mind, the West - or more precisely America - came roaring back to invade and take over Somalia.

What happened in Somalia in the year between December 1992 and the end of 1993 was an extraordinary sequence of events - driven by a new idea. It was the belief that you could invade another country, not because it threatened you or because of any power politics - but because you were bursting with good intentions to save innocent people. And it all went wrong.

The UN's failure to save victims of the famine, combined with the TV images of starving children had cause outrage in America. In the face of that a powerful group of people at the top of some of the relief agencies proposed an alternative. This group have been called "the international humanitarians" and their solution, which would have been unthinkable only a few years before,was that you should go and occupy a country militarily on humanitarian grounds.

They were driven by the sincerest of motives - but it was also going to give them extraordinary power, especially as all sorts of other groups in Washington saw in their idea new opportunities for themselves now the Cold war was over. In December 1992 a wave of political pressure built up on the outgoing President Bush. It was led by the humanitarians - like Philip Johnston, the President of CARE-US who wrote letters to the press saying bluntly:

"the international community, backed by UN troops, should move in and run Somalia, because it has no government at all"

He was backed up by powerful newspaper columnists like Leslie Gelb of the New York Times who put it more bluntly - it should be a policy of "shoot to feed" he said. The US military also joined in because they were keen to prove that they could do OOTW - "Operations Other Than War". And there were powerful elements in the State Department who pushed for it, plus the UN who were feeling incredibly guilty and wanted to be seen to be doing good. UN officials told the press that 80% of food aid was being looted - which was completely untrue. The figure was more like 20% or less.

The writer and African expert Alex de Waal has written a fantastic book that details how this rush to intervention built up in the final weeks of the Bush administration - and how in the process it fatally simplified the country of Somalia. It's called Famine Crimes - and in it de Waal says:

"By the time the drumbeat for intervention reached it's crescendo, the vision of 'Somalia' in which the US marines were intervening was wholly different from the real Somalia experienced by Somalis."

And here is a fantastic, clear and thoughtful documentary about what happened next in Somalia. It was made during the first six months after the invasion by the British journalist Richard Dowden who knows the history of Africa well. He starts by going back to Victorian times to show how the British missionary David Livingston was the humanitarian interventionist of his time. Livingston was shocked by the Arab slave trade and persuaded the British government to intervene - but that inexorably led to the European takeover of Africa.

Dowden is convinced that the Somali intervention in 1992 - Operation Restore Hope - is going the same way and leading to what is in essence an imperial takeover of the country. He films fantastic detail - like the daily morning meeting of the humanitarian aid officials and the US military. It is in all but name the government of Somalia - but as Dowden shows, it completely ignores the Somalians.

And he shows all the other factors that are coming into play. The Pentagon who are desperate to keep their budgets, the ex-secretary of Defence - one Dick Cheney - who says that is not true, contrasted with the star of the show, a US marine interviewed on the street who puts it all so clearly:

"the place is filling up with American contractors all bidding to rebuild this joint. That's all the Defence Department is. We're bodyguards for American contractors ……………… You should know that - you've been to college."

Dowden's film was made in the first six months of 1992. What he missed was another factor that came into play with the new Clinton Administration in Washington which was going to give an extra twist to the story.

The Bush administration had been persuaded to invade - but they were fundamentally conservative in their outlook. Their aim was to protect the humanitarian groups who were feeding the starving Somalians. The only problem was that by the time the US Marines got there, the famine was almost over. Faced by this there was growing pressure to expand the mission - and with the Clinton administration came a new idea. Nation-building.

This meant trying to create a new peaceful state in Somalia. But that in turn meant taking on the warlords who were causing such havoc in the country. One of the most powerful of these was General Mohamed Farrah Aidid. He was the head of one of the groups who had overthrown Siad Barre and he and his militia now dominated Mogadishu.

General Aidid was far from being a simple gangster villain - which is how he was portrayed. He had served in Siad Barre's cabinet, he had been the ambassador to India, and had at one point been Barre's intelligence chief. The problem for the Americans in Mogadishu was that Aidid thought he was their friend. When they had first arrived they had turned to him to help protect the aid agencies, had told the press what a helpful person he was, and had even rented a house from him.

But now Aidid was an obstacle to nation building. This came to a head in June when a group of Pakistani UN soldiers under US control went to search some of Aidid's buildings - including Radio Mogadishu. Aidid's men ambushed them and 24 Pakistanis were killed. At that point General Aidid began to see the UN and their American sponsors in a different light. He realised that they were like a rival clan. And a vicious four-month war began.

In the war thousands of Somalis were killed. Alex de Waal has written a very powerful piece of journalism that says the US and other troops under its control were guilty of serious war crimes in Somalia. You can find it here - and it describes in detail how the months of bitter urban warfare allegedly involved the Americans firing rockets into a building where a group of Aidid's supporters were holding a meeting - killing 54 civilians, and knowingly firing missiles into a hospital full of innocent people because they though Aidid was hiding there.

This is part of a documentary made in 1994 that tells the story of this war - and how it ended with the events just outside the Olympic Hotel in October 1993. They are the real events later told in Black Hawk Down.

In Algeria a bloody terrorist war had also begun. But this was going to last for six years.

At the start of 1992 the Algerian Army stepped in and took power in a coup. They did this because FIS had won the first round of the postponed national elections - and the army argued that they had to cancel all elections "in order to defend democracy". FIS was dissolved and more than 40,000 people were arrested and sent to camps deep in the Sahara.

For a year things were relatively quiet. But then armed jihadist groups emerged out of the hardline remnants of FIS and other Islamists. The key figure who united a number of the groups into what was called the GIA - The Armed Islamic Group - was a car mechanic from Algiers called Abdelhaq Layada.

In an interview in March 1993 Layada spelled out his theory. He quoted Oswald Spengler on the decline of the West, and Bertrand Russell on how "the white man has had his day" - and said that a new Islamist society would arise instead. He then explained how those who stood in its way should be killed.

Layada justified this by using a theory which lay at the heart of modern Islamism but which was extremely dangerous because it was so imprecise. It said that those who had become involved with western style politics and power had entered into a state of barbarism or "Jahiliyyah" and that this meant they were no longer Muslims. That, in turn, could be interpreted as meaning that they were impious, or "takfir" - and that meant you could kill them.

The danger was that there was no objective way of defining who was impious or not. Layada said that it included not just politicians, but anyone having anything to do with the politicians. So, starting in March 1993, the Islamists began assassinating intellectuals, journalists, professionals - like doctors, and academics. Many had nothing to do with the regime, but in the eyes of the young Hittistes they were hated French speaking intellectuals, and that was enough.

Then Layada was arrested. The new GIA leader was called Djafar al-Afghani - because he had fought in Afghanistan - and he broadened the category of who could be killed even further. It was correct, he said, to kill godless foreigners as well as godless Algerians. The dangerous logic of Islamism's unthought-out theories was beginning to take hold of the movement.

Here is part of a film made about the Islamist terrorist movement in Algeria. It was shot in 1994 as the killing was broadening out. The programme went and filmed the Islamist fighters in a camp in the mountains, and it is very odd. They are all posing for the cameras, and it has the mood of a holiday camp. They all lie around watching TV, then get up to send death threats by fax, and settle down at the sewing machine to stitch leather bullet belts.

The weirdest bit is where "the bomb-maker" shows how he makes a giant bomb using a gas canister. It's like a grotesque cookery programme - even down to the seasoning he drops in at the end - horrible small bits of metal shrapnel to make it kill more people. Come die with me.

Then in September 1994 something very strange happened to the GIA. After two of it's leaders were killed, a new leader was appointed called Djamel Zitouni who was the son of a poultry merchant.

He escalated the violence - first of all he started a terrorist campaign in France. Then he followed the logic of who could be killed to include a new category. Zitouni declared that all the other Islamist factions that didn't agree with him were also godless - and so they should all be killed as well.

It didn't stop there. Throughout 1995 there were purges within the GIA itself - which seemed to mean that Zitouni had decided even those around him were impious too. His rivals in the GIA, and a number of journalists, were convinced though that what was really happening was that the Algerian intelligence agencies were manipulating him. That the intelligence agents were cleverly twisting the mad logic of Islamism so that it would end up destroying itself.

Or maybe not. Because it got even weirder and more horrific. In July 1996 Zitouni was shot and Antar Zouabri took over.

The GIA then began to kill hundreds of ordinary Algerians in a series of horrific massacres - culminating in terrible bloodbaths in August and September. The GIA put out a communique taking responsibility for the killing and saying it was justified because everyone in Algeria who didn't join the GIA was now impious and thus should be killed. As the historian of Islamism, Gilles Kepel, put it - the GIA had decided on the excommunication of the whole society. Except for themselves, everyone else must be killed.

It was mad, and the Islamist movement in Algeria fell apart.

Here are some extracts from a report on the horror that was being created both by the terrorists and by the government. It begins with a woman called Zora who is a very perky PR for the military rulers. What follows shows the reality.

At the heart of both these stories - in Somalia and in Algeria - is is a simple question. Could the chaos and the horror have been avoided? Was it inevitable?

In both cases there are fascinating clues.

In Somalia the evidence lies in a place that, as far as the rest of the world is concerned, doesn't exist. It is the independent Republic of Somaliland - which is the large northern part of Somalia that in 1991 seceded and set up as an independent self-governing state. Since then the people of Somaliland have built - from the bottom up - a safe, successful and democratic state without any foreign aid or outside intervention.

It is a fascinating story that has gone almost completely unnoticed and unreported. After 1991 thousands of Somalis who had fled abroad came back and began to set up in the new state all sorts of individual business initiatives that gradually rebuilt the state without any centralised guidance. They became known as "The Somaliland Pioneers". Then out of that emerged a new idea of how to take the existing Somali clan system and integrate it with the idea of multi-party democracy. This idea itself was developed democratically out of a series of grassroots meetings - not imposed by western 'nation-builders'.

The journalist Mary Harper is the BBC World Service African Editor and she has written an absolutely brilliant book about Somalia and its recent history. Anyone who wants to understand what has really gone on in that country should read it. It's called Getting Somalia Wrong and in it she is absolutely clear about what the story of Somaliland means:

"Because Western models of peacemaking and state-building have not been imposed from the outside, Somaliland has in many ways saved itself from the fate of Somalia. The example of Somaliland has demonstrated that, when left to themselves, Somalis can form a viable nation state.

After breaking away from Somalia in 1991 the people of Somaliland looked deep into their own traditions, building a system, which was initially based on clan politics, but over time incorporated more modern political institutions and processes. A hybrid system of government was designed, whereby Western-style institutions were fused with more traditional forms of social and political organisation.

And it was rooted in a popular consciousness rather than imposed from above."

This is the government of Somaliland - a country that is unrecognised by every other nation in the world

There are also practically no BBC television reports about what has happened in Somaliland - except for a film made by the intrepid reporter Simon Reeve in 2005. He went to look for this place that everyone in authority said didn't exist. It's a great film full of great facts. At one point he visits an airport that was originally built by the Soviet Union - which has the longest runway in Africa. The Americans then later used it for the most unexpected reason.

In Algeria the clue as to what might have happened lies buried in the chaos of 1991 when the Islamist Party FIS was heading for power.

In the first round of the national elections in December 1991 FIS lost nearly a million votes compared to the local elections the year before. FIS still won comfortably - but the big drop in support showed that the Islamists had begun to frighten and alienate large sections of the Algerian middle classes. The historian of Islamism, Gilles Kepel, has argued that this meant that FIS had passed its peak as a political force.

No one can know if that was true - but if FIS had been left alone they would have had to face the fact they were alienating a very powerful section of the urban middle classes in Algeria. Then they might have had to come to terms with the realities of power in complex modern societies.

Instead they were forced out into the barren wilderness - both literally, and in their imaginations. This led to violent illusions. A simplified vision of Algerian society took hold of the Islamists' minds - divided between good Hittistes and the bad westernised elites who had been poisoned by democracy. And that led them to believe they could use violence to force the kind of society they wanted into existence.

But maybe the same thing happened to the West in Somalia with the humanitarian intervention?

In 1992 a loose group of humanitarian internationalists, policy wonks, politicians and TV journalists invented a simplified vision of Somalia. It was a country full of innocent dying children and evil warlords riding around on their technicals. This was a picture almost completely detached from the complex questions of power that were causing such chaos in Somalia.

Then, when things didn't go as simply as the humanitarian vision predicted - and the westerners got inevitably caught up in the local struggles - they turned to violence to try and enforce a simplified vision of democracy and nation-building on the country.

And we still haven't learnt.

For ten years after 1994 Somalia descended into chaos. It wasn't caused by the failed western intervention - the Somalis bear a great responsibility. But then, in 2005 an Islamist movement emerged called the Islamic Courts Union which began to impose a new kind of order and stability in areas of the country using sharia courts. Mary Harper, and other journalists who know Somalia, have argued that the Islamic Courts were a local grassroots attempt to create order and essential services - similar to what had happened in Somaliland to the north. It's important not to romanticise them, but they were a fragmented and essentially local version of political Islam - with the violent extremists very much in the minority.

America's response was to immediately label the ICU as yet another part of a global jihad network linked to "Al Qaeda". Within six months Somalia was invaded by Ethiopia - backed by America and supported by giant US gunships and a naval fleet. The ICU fled - but then the Islamist movement re-emerged in a much more violent form in the shape of a group called Al Shabaab. Mary Harper says that America's intervention in Somalia had created the very thing it feared.

And the same thing is happening now all across the northern part of Africa. In Mali, in northern Nigeria with Boko Haram, and in Algeria with the remnants of the GIA. In every case what are local struggles for power are being simplified by Western politicians and commentators into part of a global battle against "Al Qaeda".

It is true that there are extreme Islamists involved who proudly announce that they are joined together into a global movement. But the reality is that that kind of extreme Islamism has failed everywhere. Ever since Algeria in the early 1990s none of the extremist salafist-jihad groups have managed to take power and create the kind of society they yearn for. The reason for their failure is simple - the growing urban middle classes throughout the Arab world don't want it. You only have to look at the battles now tearing Egypt apart to see that happening.

Instead our politicians and allied terror experts fall for the Islamists' attempts to aggrandise themselves - and in the process become the Islamists' PR agents. It means the western elites are helping to promote a failed revolutionary movement while ignoring the signs of what might be the future for Africa - the new systems of multi-party democracy being built from the grassroots in places like Somaliland. Without aid, and without the west imposing centralised forms of control.

Meanwhile most western aid agencies working in Africa have a very firm policy. They do not talk to the press or TV any longer. They keep what they are doing completely secret. Given what happened in places like Somalia it is a very sensible policy. But it leaves us and our leaders ever more lost in the wood looking for the baddies hidden behind the trees.

Animals have been a central part of television from the very beginning. But over that time the way animals are portrayed on TV has varied enormously - not just in the way they are filmed, but in the stories they are used to tell the viewers.

And the truth is that the animal programmes are far more about us than they are about the animals. They are really about how we see ourselves. I have always been convinced that animal programmes are one of the most powerful ideological expressions of our time - telling stories that both express and reinforce how we understand our relationship to each other socially and politically in powerfully emotional ways.

Over the past thirty years the wildlife programme has been dominant, led by David Attenborough. The story these programmes tell is a deeply conservative one. The central, natural, unit that the films portray is the family - and they tend to follow that social unit through repeated cycles of birth, discovery, danger and tragedy - followed by the birth of the next generation who will repeat the cycle.

The backdrop to this story is the endless repetition of the seasons - "spring returns and the first green shoots force their way through the melting snows" - which gives the cycle a natural inevitability that reflects and echoes back to us the static conservatism of our age.

But it wasn't always like this - and for Christmas I want to tell the story of the far more larky and chaotic age of animal programmes that came before in the 1970s and early 1980s.

It is The Age of the Talented Pet. It was a way of portraying animals on TV that was not only very funny - but was also equally a powerful ideological expression of the politics and aspirations of the time. I don't think this has been properly recognised and I would like to set the record straight.

To put the age of the talented pet programme fully in context it is necessary to start with the way television portrayed animals - and pets in particular - before that, in the 1960s.

I have found in the archives an absolutely wonderful film made in 1969 about the relationship between pets and their owners. It is called Love of a Kind, and it is a series of scenes and stories about different owners and their pets. Some are very funny, others are odd and eccentric, and some are incredibly moving.

It is a really good film that is also brilliantly shot in that loose 1960s verite way where you get the feeling that the camera is just looking around as a normal person would. It also perfectly expresses the belief that underlay the counterculture notions of the 1960s - because at heart it is about eccentricity and tolerance of oddness and difference.

Everyone in it - whether animal or owner - is a distinct character who is being just what they want to be. I particularly love Flo the enormously fat and very grumpy cat who begins the film, and Benji the vicious Cairn terrier who just goes for everyone - including his owner and her close friend. Their dialogue as they discuss why Benji does this is great.

The film is about how individuals and animals can forge deep emotional relationships - yet still fully be themselves in all their awkward and grumpy ways. It shows these deep bonds in some incredibly moving ways. The scene where an old woman waits while her dachshund is operated on by the vet is just heartbreaking and so moving.

But then, at the end, the credits reveal that the film was shot and directed by Lord Snowdon - and you can't but help get the feeling that what the film is really expressing is a traditional One-Nation Tory fantasy about the world - where everyone can be funny eccentrics and be happy, providing that they all know their place on the estate. Perhaps that was always the idea that underlay the hippie dream.

The problem was that by the end of the 1960s more and more ordinary people didn't want to be patronised by the upper middle class elites in Britain and kept in their place. They didn't want to be told what was the right way to think and behave - because that somehow implied that the elites knew what was right, and so were cleverer than everyone else.

This rebellious feeling rose up among many ordinary people in the 1970s and would later be co-opted by the right under the term "aspirational". At its heart was a conviction among those people that they were just as clever as the patronising elites.

And as this feeling rose up so did a new type of animal programme on British television. Talented pets were animals who wanted to be as clever as their owners and took great delight in showing that they could do many of the things that humans could - like talk or sing or dance or even skateboard.

Here is one classic example. It is Meg the Counting Dog and her owner Mrs Martin. And Meg can not only count, she can do a lot more mathematically. Aspirational Dog.

From one perspective these short films - which were predominantly made by the programmes Nationwide and That's Life - can be seen as deeply patronising to the owners of the animals. But they didn't patronise the animals - what comes over in most of them is the sheer joy and liberation that the animals clearly feel as they behave in sometimes the silliest ways - just like humans.

Here is the one that I think is both the oddest and the funniest of all these short films. I don't want to give anything away except to say that I call it The Soda Dogs - and it makes me cry with laughter, above all because of the sheer eagerness and excitement on the dogs' faces.

And the animals got cleverer and cleverer. Here is one of the great talking dogs. He is called Domino. He only has one phrase but the film brilliantly repeats it in inventive ways. And the phrase is also a perfect expression of the world of 1980s and 90s consumer aspiration that about to come.

And notice how the power structure was shifting. Along with the talking dog is the non-talking husband, sitting next to his wife on the sofa. His only job is to feed biscuits to the dog as it talks.

But as well as being odd expressions of the new aspirations of the time, these films also express the sheer anarchic silliness of the late 1970s and early 80s.

I think that that silliness was one of the products of the economic collapse and political chaos of the post-war planned society - a free-wheeling individualism born out of a general realisation that the elites who were in charge didn't have a clue any longer about what was going on. And it was by no means inevitable that the right would grab hold of that individualism. If the left had had the imagination and courage - they too could have taken hold of it and steered Britain in a completely different direction.

And here is a collection of the best of these silly, talented, anarchic animals. The wonderful somersaulting dog, plus Shep the dog that that's going to play Salut D'Amour by Sir Edward Elgar on the piano the way he wants to - in a fabulous out-of-tune style, and the singing parrot who accompanies his policeman owner in Weymouth.

But in amongst all this new-found self-confidence among the pets of Britain there were still the ghosts of the old rigid owner-pet power structure.

Here is a beautiful moment I discovered in the live Election Programme from October 1974. It is 6.30 in the morning and the programme goes live to Downing Street. It is deserted except for one old man who is waiting to welcome Harold Wilson back as Prime Minister. With him is his dog - waiting mutely as his owner is interviewed, not allowed to do anything. He knows his place - very Old Labour.

(To be honest I have also put this in because the interview is great and the man's explanation of why he is there is beautifully logical and deadpan. Again very Old Labour)

And the talented pets got weirder and also began to exploit their special talents. Here is a short film about Balls the Bat - plus his owner Cherry Bramwell. The bat is beautiful - and I love the bit where it goes shopping - but he has begun to go commercial, having just had a starring role in a movie.

And you can also see how the old power structure between owner and pet is beginning to be reasserted by the owner. Cherry is a brilliant interviewee - because she has realised the basic law of all comic factual TV. That if an interviewee is serious about an absurd situation - they are funny. If they think it's funny - it's not funny at all.

Cherry is deadpan serious and thus funny. But in reality she is acting. The earlier innocence of the talented pets' owners is disappearing to be replaced by a controlled reality.

I want to end with a legendary moment from the Age of Talented Pets. It is a moment that millions remember - but it also shows dramatically how the silliness and anarchic stupidity was now beginning to be managed and controlled.

It is Prince, the dog from Leeds that said "Sausages". It is very funny – but, as the owner Paul Allen admits, he is manipulating Prince's throat to make the words. He has an elaborate justification for this - but, like a flash of lightning on a dark night, it shows how the individualism of the talented animals was now being increasingly institutionalised and managed. The age of innocence was over - and you could see the reality of what Thatcherism was going to become.

Then - in the 1980s - the talented pets receded in TV. They still exist, like Pudsey the dancing dog and Simba from Top Dog model, but their place at the top table of TV culture was taken by the epic, conservative moral stories of the wildlife programmes and series.

We have lived with that portrayal of animals for thirty years, mixed in with programmes like When Animals Attack - that was started by Fox TV in the 1990s, that also has an implicit conservative message - the eternal law of the jungle.

But maybe that age is coming to an end as the boosters for our conservative age sound ever more uncertain. And at the same time the animal programming on the BBC is weakening and being challenged by the kingdom of Youtube with its wonderful range of stupid animals doing very silly things.

If animals on TV are the innocent ideological expressions of our age - maybe it is possible to look to the sneezing panda and its allied operatives on Youtube as the harbingers of what is to come. The return of the revolutionary libertarianism that was glimpsed with the joyous, anarchic talented pets of the late 1970s, before that moment of silly freedom was co-opted by the forces of reaction and market conservatism.

And the first hero of this new breed is Loukanikos - the dog that turns up to all the riots in Athens. Naturally his name translates as Sausage.

Last week there was yet another cycle of horrific violence in the Gaza strip. This week there are demonstrations in Cairo driven by fears that the revolution is being hi-jacked by the Islamists. Liberals in the west look on baffled and horrified. What they thought was a glorious revolution in the Arab world is morphing into something they don't understand. While Gaza is like some brutal other planet forever possessed by hi-tech assassinations and bearded aliens dragging corpses around the streets on motor cycles.

All this is comprehensible though - but only if you look at it in a wider context. A context that western liberals really don't like to think about because it makes them very depressed. It is the great shift of our time - the collapse of the dream that politicians could change the world for the better. A dream that was replaced by a conviction that politicians were untrustworthy and always become corrupted by power.

The collapse of that optimistic vision of what politics could achieve then left the way open for powerful, reactionary forces to take power who don't want to change the world. Instead they want to manage the world and hold it stable - backed up by the threat of violence. A threat to which they have become increasingly addicted.

This has happened not only in America and in Britain - but all over the world. And I want to tell the story of how it happened in the Middle East. It is the intertwined story of the rise of the Muslim Brotherhood in Egypt, Hamas in the Gaza strip and the reactionary right-wing nationalist groups in Israel.

All three groups are driven by an angry, pessimistic vision of the world, of human nature - and the inability of politicians to transform things for the better. It's a fascinating story because it shows how the underlying similarities led those groups to become tightly locked together - helping each other cement their ruthless grip on their people - and freeze out any progressive alternatives.

The story begins nearly a hundred years ago with one of the great examples of how you can never trust politicians.

The British promised the Arabs that they would create a new and better world for them. The only problem is that they promised the Jews the very same thing.

In 1915, at the height of the First World War, Sir Henry McMahon, the British High Commissioner in Egypt made an agreement with the Emir of Mecca. It said that if the Arabs helped the British overthrow the Turks who ruled Palestine - then the British would in return give the Arabs independence. Lawrence of Arabia - TE Lawrence - was one of the British agents sent to help organise the Arab Revolt.

But two years later later the British Foreign secretary, Arthur Balfour, promised the Zionist movement that a permanent Jewish homeland would be set up in Palestine. Zionism was in many ways a utopian movement. It had been invented by Theodor Herzl in the 1890s, and he believed that a Jewish state would not just rescue Jews from persecution, but it would also transform them. The state of Israel would be a new kind of environment which would turn its people into stronger and better kinds of human beings.

The British didn't care about that kind of thing. They were desperate to get America into the war on their side - and one of the reasons for the Balfour declaration was to curry favour with the Zionists and their supporters in America.

Here is part of a massive TV series made in the 1960s called The Great War. It tells the story of how in 1917 the British came to find themselves marching into Gaza (what is today the Gaza Strip) on their way to conquer Jerusalem - and the nightmare that trapped them in that small strip of land.

It also gives a very good sense of the background pressures that led Britain to making the contradictory promises.

In the 1920s Britain took over the running of Palestine and came face to face with their hypocrisy and deceit.

On the one hand Jewish immigrants began to arrive in their thousands, buying up the land from the old Palestinian families. While the Arabs were furious at what they saw as British treachery and a revolt began to grow against both the British and the Jews.

One of the main leaders of the Palestinian Arab revolt was Sheikh Izz al-Din al-Qassam. He is forgotten in the west today - but not by Palestinian Arabs and above all by Hamas who see him as the first true Islamist revolutionary. The thousands of Qassam rockets that were fired from the Gaza strip last week are named after him, as is Hamas' military wing - the Qassam Brigades.

Qassam had studied at Al-Azhar university in Cairo and had become one of new wave of reformists who argued that Islam should be cleansed of all the rituals and superstitions that had grown up over 1200 years. It could then become a powerful faith that would deal with all the modern forces at play in society - economic and scientific and political.

And he believed it could help lead a revolt against British power and the Jewish immigrants. Qassam went to the city of Haifa and began attracting followers - promoting the idea of a jihad against the occupying powers. You couldn't trust the old families who run Palestinian society, he said, because they had sold out, as had the politicians and the traditional religious leaders.

The Palestine Post recorded one of Qassam's speeches ending angrily: "Jews do not have to take the country by force as the Arabs are selling it to them"

Here is part of a film that gives a powerful sense of the strange world that Qassam was fighting against. It is about one of the surviving members of a grand Palestinian landowning family. She is called Malika Shawa - and when the film was shot in the late 80s she was running the only hotel in the Gaza Strip, playing the piano as all around her the first intifada was erupting.

The film shows how involved the Palestinian elites had become with the British rulers. Malika tells of her time being educated at Cheltenham Ladies College.

In the 1930s Qassam formed The Black Hand Gang. He and a group of followers took to the hills and for five years they launched armed attacks on Jewish settlements and on the British military and police.

The British called him "The Brigand Sheikh" and he became a terrifying figure - it was said that he would send his followers to kill anyone that said anything bad about him. But in November 1935 the British cornered him in a cave and Qassam was killed in a violent shootout.

It is important to realise that Qassam saw politicians as part of the problem. Like Hassan al Bana who had founded the Muslim Brotherhood in Egypt in the 1920s with the slogan "The Koran is our Constitution", Qassam saw modernised Islam as a total system that could replace politics. You had to do this because if you left politicians to their own devices they lied and betrayed you, as the British had done, or sold you out, as the Palestinian elites were doing.

In contrast, the Zionists who were moving into Haifa and the rest of Palestine in the 1930s believed deeply in the power of science, technology and politics to change the world for the better. Many of them had read a novel written by Theodor Herzl in 1902 called Altneuland - Old New Land.

The novel is a utopian vision of a future perfect society set up in Palestine with the city of Haifa at it's heart - Herzl calls it "The City of the Future." Herzl's Zionism was part of a socialist vision of utopia that went back to writers like Fourier and Saint Simon, and he described a society where the land was under common ownership and people lived in co-operatives and communes. There was also a model welfare system, no social classes and exploitation - yet individuals could pursue their own ends and profit by them.

It was a glorious vision, but it was also firmly rooted in the European tradition of empire. In the novel the characters listen to a phonograph roll that describes the achievements of The New Society for the Colonisation of Palestine. It describes how the benevolent technocracy that runs this new society has brought the benefits of European progress to a backward and sparsely populated land.

That's not quite how Sheikh Qassam and his Black Hand Gang saw the Jewish settlers.

But then - in the late 1940s - a new political force emerged to challenge Zionism - Arab nationalism.

It's charismatic leader was the President of the new independent Egypt - Gamal Abdel Nasser. He became a heroic and inspirational figure for millions of Arabs because he promised a united Arab world that would become strong enough to challenge western imperialism.

And also strong enough to challenge the new state of Israel which had been established in Palestine after the war of 1948 between the Arabs and the Jews.

Here is Nasser talking about the revolution he has begun - and you get a good sense of the progressive optimism at its heart. Nasser was convinced that the Arab people could be transformed by a modern planned socialist society into new, more confident individuals who would no longer meekly accept the iron hand of authoritarian dictators who were backed by the west.

Nasser was ignoring the fact that he himself was an authoritarian dictator.

And Nasser began to organise the fight against Israel - using the Gaza strip as the base.

After the 1948 war hundreds of thousands of Palestinians had ended up living in refugee camps in Gaza. Beginning in 1955 Nasser got Egyptian intelligence to organise small resistance groups from the Palestinians in the camps. They were called Fedayeen and they started to do hit and run attacks into Israel.

Israel retaliated by attacking the Gaza strip and the border guards. The man who led the commando units that fought back against the Fedayeen was a young Ariel Sharon.

Here are the fragments of film from the archive that report those incidents. What is really interesting is how forcefully both America and Britain in the UN condemn the Israeli actions.

At first the Islamists - the Muslim Brotherhood - welcomed Nasser. They liked the fact that he banned all political parties because it seemed to fit with their ideas about the "unity of the faithful". But they quickly discovered that Nasser's idea was to turn Egypt into a modern secular society - inspired by socialist ideas and driven by the by the ideology of authoritarian nationalism.

So they tried to assassinate him. In turn Nasser jailed or hanged several of their leaders and sent the rest into exile. Everyone thought the Muslim Brothers were finished - another hangover from colonial times gone forever.

I have found a fascinating film in the archive which shows how dramatically marginalized the religious establishment became under Nasser. It's about the famous Muslim University at the grand Al Azhar mosque in Cairo. For centuries it had been the powerhouse of Islamic thought throughout the Arab world. It was the place that Sheikh Qassam had gone to study - and where he had become inspired by the new radical ideas of reforming Islam.

But now Al Azhar was under orders from the revolution to modernize in a very different way. The film shows how the revolutionary government has insisted that Al Azhar teach courses that have nothing to do with religion - even a department of Business has been formed.

And the new, modernising head of Al-Azhar says he is trying to prevent a class of "priests" arising who will stop progress.

In the 1950s Israel was also driven by a deep sense of progressive optimism. And in an odd way it mirrored the ideas of a planned socialist society that Nasser was trying to build.

Starting in the 1930s, the Israelis set out to try and build in Palestine the new kind of Zionist society that Theodor Herzl had laid out in his novel Altneuland - Old New Land. The new capital was called Tel Aviv - which was the Hebrew title given to Herzl's novel by it's translator. It roughly means "a new spring coming from an old mound".

The new city was constructed as a grand experiment in town planning. It was based on plans drawn up by the Scottish town planner, Patrick Geddes. His ideas about how cities could be planned came from the same utopian traditions as Herzl's belief in a socialist planned society. What linked them was the technocratic belief that flourished in the 1930s - and again in the 1950s - that you could shape the environment around human beings as a total system that would make them stronger, more confident and morally better human beings.

It was a grand dream. Here is Patrick Geddes.

And here is the utopian city that was built according to his plans - it was called "The White City". Many of the architects who actually designed it had been trained in the 1930s at the Bauhaus school and were deeply influenced by the ideas of Le Corbusier. One pamphlet described the ideas behind it:

"The city is an experimental laboratory for the implementation of modern principles of planning and architecture, it has influenced the whole country.

The plan was based on the idea of creating a new place for a new society, where the Zionist ideal would come true through the Modern Movement. It is also a synthesis between Oriental and Western cultures."

And in the 1950s - that utopianism spread through a lot of Israeli society . At it's heart was the kibbutz movement. Again the idea of the kibbutz had been developed in the 1920s - and was an attempt to create model socialist collectives that were a concrete expression of the Zionist theory.

The kibbutzim were more than just a collective way of managing the land. They were seen as a new kind of environment in which individuals would come together in the evenings, have group dances and then group discussions. In some cases the discussions were like early versions of group therapy - individuals being given permission to express their ideas and feelings. Out of all this would come "new people".

Unfortunately many of the kibbutzim had been constructed on land on which Palestinian Arabs had lived - and whose families now lived in cramped misery in the refugee camps in the Gaza strip. And increasingly there was a realisation in Israel that the kibbutzim were a powerful weapon in establishing a more permanent Israeli presence in the outlying fringes of the new state.

The kibbutzim in the 1950s and 60s became a weird combination of happy-clappy utopian socialism and an armed fearfulness - with bomb shelters and trenches built around their modernist-inspired communal halls. It was a bit like some JG Ballard story.

But then a character from the past came back in a dramatic way into the heart of Israeli society. And his presence - and what he said - sent out shockwaves that began to undermine the very underpinnings of the optimistic progressivism at the heart of Israeli society.

In May 1960 a group of Mossad agents kidnapped Adolf Eichmann in Argentina. They drugged him and flew him to Israel on an El Al plane disguised as a member of the plane's crew.The kidnapping was a world-wide sensation because Eichmann had been one of the main organisers of the Final Solution - the mass extermination of the Jews.

A year later the Israelis put Eichmann on trial in Jerusalem. He was encased in a bulletproof glass booth - and it became a powerful image of this terrifying figure who had organised the Holocaust sitting on show in the midst of the new state of Israel.

A number of historians have argued that Eichmann's trial created an enormous shock to Israeli society because for the fifteen years after the second world war no one in Israel - or in the Jewish communities in America - really talked about the Holocaust. It was if it was forgotten and wiped.

Hundreds of thousand of survivors from the death camps came to Israel, but the mood among them was to look towards the future - turning their faces towards a better future promised by the Zionist dream, and trying to forget the horrors of the past.

Above all they didn't want to be seen as victims in an optimistic age. The leader of the American Jewish Committee wrote that

"Jewish organizations should avoid representing the Jew as weak, victimized and suffering" Because it reinforced "long ingrained stereotypes - the hunted wanderer, inured to universal hatred and contempt"

Other historians have challenged this argument - and it can quickly lead into the dead end of arguments about how the memory of the Holocaust has been used and abused.

But I have found a really interesting film shot in Israel in 1961 during the Eichmann trial. It asks ordinary Israelis - including some on a kibbutz - what they feel about Eichmann and his effect on their world. Some approve - but the majority feeling is that this should have been forgotten - and is doing real harm to the new country of Israel.

One woman who speaks very powerfully finishes - "I would be happy if he had never entered this country"

But that was only the beginning of the terrible corrosive effect Eichmann was going to have not just on Israeli optimism about their society - but on the whole western liberal belief that human beings could be transformed for the better.

In 1963 a political philosopher called Hannah Arendt who had attended the Eichmann trial published a series of articles in the New Yorker. In them she challenged the idea put forward by the Israeli prosecutors that Eichmann was a special kind of evil human being. Arendt argued that he was the very opposite - that he was "terrifyingly normal". That far from being a demonic monster he was actually a bland, mindless and extremely efficient bureaucrat. He was motivated, she said by personal ambition and that he wasn't even particularly anti-semitic.

Arendt called it "the banality of evil".

"evil deeds, committed on a gigantic scale, which could not be traced to any particularity of wickedness, pathology, or ideological conviction in the doer.

However monstrous the deeds were, the doer was neither monstrous nor demonic.

Evil can spread over the whole world like a fungus and lay waste precisely because it is not rooted anywhere. It was the most banal motives, not especially wicked ones which made Eichmann such a frightful evil-doer."

Arendt's reports caused an outrage. The journalist Norman Podhoretz wrote that Arendt's picture of Eichmann -

"violates everything we know about the Nature of Man."

And that went to the heart of it. Because what Arendt was implying was that human beings might not be changeable or perfectible. That anyone could do really evil, horrible things any time depending on the circumstances they found themselves in. And what was worse - that the modern world of intricate bureaucracies and bland management might make it more possible.

It was a pretty pessimistic and conservative view of human beings - and it challenged the idea that you could change the world for the better. And this dark frightening idea, born out of the horrors of twenty years before, began to worm its way into the post war optimism not just in Israel but a whole generation of liberals in Europe and America.

Here is part of a documentary about Arendt and the trial of Eichmann. The first interviewee is Arendt's biographer, the second is one of her students. They are intercut with the extraordinary defence Eichmann gave at the trial. He does sound like a General Manager trying to excuse himself.

And four years later the optimistic vision of the future that Nasser had held out to the Arab people also began to collapse - because of Israel.

In June 1967 Nasser was told by the Soviet Union that Israel was planning to attack Egypt - so he began to mass troops. The report was false - but in response the Israelis launched a pre-emptive attack against Egypt and Syria.

It was a catastrophe for the Arab states. In six days Egypt's military was overwhelmingly defeated. It was also a crippling humiliation for Nasser because it exposed as a sham his promise that the Zionist state would be annihilated. Nasser then behaved like a petulant drama queen - resigning in a spectacular public way, then retracting it.

Millions still loved Nasser - but the defeat was the beginning of the end of the dream that a new confident Pan-Arabism could transform the fortunes and the subservient psychology of the Arab people. Left wing students began to protest in Cairo - they demanded Egypt attack Israel again, and they blamed the defeat on corrupt generals who headed the Egyptian Army.

But power in the struggle with Israel was now seized by the revolutionary left in the Palestinian refugee camps. In 1969 Yasser Arafat became the head of the Palestinian Liberation Organisation which was an umbrella for a range of left-wing, secular groups - including Arafat's own Fatah organisation.

And again Gaza was the centre of the opposition. After the Six Day war Israel had taken over both Gaza and the Sinai peninsula and the Palestinian refugees now found themselves facing the Israelis as their overlords.

Here are some of the earliest news reports from Gaza about the new young terrorists who are promising to rid Palestine of the state of Israel. It begins in 1969 with the coverage of three Palestinian schoolgirls who have been arrested and put on trial for supporting "a subversive organisation"

It is all a bit ramshackle, and the journalists have no idea really what they are reporting on. Then I have added a report from just three years later - 1972 - about an Al-Fatah training school for children. It shows just how quickly the movement has grown - and how intense the belief in the armed struggle had become.

Although Nasser's dream had failed - and he died in 1970 - the PLO and their fighters had inherited his progressive world view. Many of the groups in the PLO were left wing revolutionaries and they believed that they were not only fighting to get rid of Israel, but also to create a new kind of secular, socialist state in Palestine.

But in Egypt that optimistic view of politics and its ability to transform society was collapsing. A vacuum was opening up which would be filled by the group that only fifteen years before everyone thought was dead and buried - the Islamist Muslim Brotherhood, along with their much more conservative view of how to run society.

In 1975 a feature film was made called Al-Karnak. It told the story of how after the defeat in 1967 hundreds of Nasser's opponents had been jailed and tortured. The film showed the torture in detail and it was a powerful exposure of how Nasser's visionary ideals had become horrifically corrupted.

It seemed to prove dramatically the central message of the Islamist movement - that if you gave power to politicians in a secular society they would inevitably become corrupted and dangerous - however noble their original ideals had been.

Here is part of a documentary made in Cairo as the movie gripped both the elites and ordinary Egyptians. It begins with Mustafa Amin - a famous journalist who had been one of those imprisoned and tortured. Then it goes on to the sensation caused by the movie. It is a good report because it gives a real feeling of the changing mood within the Arab world at that moment in the mid 1970s

And that pessimistic mood began to spread through the Palestinian resistance movement too - carried by the odd logic of terrorist violence. Because the terrorists' actions would lead them to be haunted by the same old ghost that Eichmann had brought back into the heart of Israel - the Final Solution.

Since the early 1970s various different Palestinian groups had hi-jacked western passenger planes. The motive was to draw attention to the plight of the Palestinian people and their fight against Israeli occupation. They also had developed close links with a number of western terrorist groups - in particular the groups in West Germany like the Red Army Faction, and The Revolutionary Cells.

In June 1976 a group of terrorists hi-jacked an Air France plane and flew it to Entebbe in Uganda. Some of the terrorists were from the Patriotic Front for the Liberation of Palestine, others were from the German Revolutionary Cells. At Entebbe the terrorists began inspecting the passengers' passports. As they did so they separated the Jewish passengers out from the others, and said they would release the non-Jewish hostages.

It was a powerfully symbolic moment for the revolutionary left - both Palestinian and German. They had turned to violence in the belief that they were fighting to go forwards - to liberate Palestine and create a new revolutionary world. Instead they now found themselves behaving like the Nazis thirty years ago separating the jews out from the others.

One of the Jewish hostages later described how he had shown the terrorists the concentration camp number tatooed on his arm. He described how one of the German terrorists, Wilfried Bose, plaintively responded - "I'm no Nazi - I'm an idealist"

It seemed that idealism might be taking the secular revolutionary movement not forwards into a better future but backwards into the very worst times of the past.

It is also important to remember that one of the Israeli rescuers who was killed at Entebbe was Jonathan Netanyahu. He was the older brother of Benjamin Netanyahu - the future Prime Minister of Israel. His brother's death, it is said, was a powerful shaping force on the younger brother.

By the late 1970s there was a massive political, social and moral vacuum at the heart of Egypt - and much of the Arab World. The collapse of President Nasser's grand progressive, and secular vision had left the society adrift.

Into the vacuum came a resurgent Islamism. Some of the Islamists turned to extremism and violence - like the Al-Jihad group who assassinated President Sadat in 1981. But the Muslim Brotherhood took another route.

Sadat had freed many of the leaders of the Muslim Brotherhood from jail, but they were banned as a political party. So the leaders of the Brotherhood turned to building their influence through the complex social and professional organisations in Egyptian society. Brotherhood members stood for election to the syndicates and guilds of many of the leading middle-class professions - and in the 1980s they took control of the doctors, the dentists, the engineers, the pharmacists, and even the Egyptian bar - the lawyers.

At the same time the Brotherhood created a powerful system of social welfare for millions of ordinary Egyptians in villages across the country that was far more efficient and responsive than the cumbersome state welfare.

Many middle-class Egyptians began to fear a silent, creeping political coup. But the Brotherhood argued that what they were doing was openly creating the foundations for their idea of a modern society. Islam would be a total system that could manage and guide all parts of society.

Here is part of a film made about the rise of the Muslim Brotherhood in Egypt. It was filmed in 1992 and it is really good because it takes you into the heart of their revolution and allows them to express their utopian vision. But it is a deeply conservative sort of utopia - because the system they want to build would act as a restraint on politicians who tried to use their power to change the world. You couldn't let them do that because it always led to disaster.

I love the TV preacher who argues that society is like a TV set. God, he says, is just like the person who writes the instruction manual for a TV set.

"The rules are written by the person who creates it.

And when it goes wrong you take the product to the manufacturer. He knows how to fix it. But if you take it to someone else he screws it up."

That puts politicians in their place.

And the prizes given by the Muslim Brotherhood's newspaper for their religious quiz are great. First prize - a trip to Mecca. Second prize - a vacuum cleaner.

I have followed it with part of another documentary about how the Muslim Brotherhood took over the Lawyers Syndicate. Their opponents forcefully argue that this is a silent, creeping political coup. The two films take you to the heart of the mystery about the Muslim Brotherhood. What are they really up to?

At the same time the Muslim Brothers' ideas - and their techniques - began to spread into the Gaza strip. And as they did so they became weirdly mixed up with the Israeli forces who were fighting against Yasser Arafat and the PLO. Out of that would come a tacit cooperation to destroy a common enemy but it would also have very dark consequences - it would lead to both sides becoming locked together in a static world.

It happened through the rise of Hamas - who were directly inspired by the Muslim Brotherhood.

To begin with they weren't called Hamas. Back in 1973 a preacher in the Gaza Strip called Sheikh Ahmed Yassin formed an organisation called al-Mujamma al-Islami (The Islamic Centre). Yassin wanted the organisation to spread the ideas of the Muslim Brotherhood through the Palestinian world - and that also meant getting rid of the secular resistance movement and replacing it with one inspired by Islamist ideas.

Sheikh Yassin was an extraordinarily powerful character. Crippled since his childhood by a broken spine he was totally dependent on his followers to look after him, feed him and put him to bed. But he inspired those around him to believe that one day their tiny group could destroy the leftist infidels around the PLO and take control of the Palestinian movement.

The Mujamma did what the Muslim Brotherhood were doing in Egypt. They set up a complex system of welfare in Gaza, including kindergartens, free food and clothing. It also set up clinics offering free healthcare and medicines. They also began to take over many of the professional associations - like the Medical Association, the Engineering Association and the Bar Association.

And the Israeli authorities not only allowed them to do this - but encouraged it. They did this because they saw the conservative ideas of the Islamists as a potent force that could undermine and damage the secular Palestinian revolutionary movement.

There is a really good book about the rise of Hamas by Beverley Milton-Edwards and Stephen Farrell. In it they got a number of very senior Israelis to admit the tacit support they gave to Yassin and the Mujamma. One director military intelligence says:

"At the beginning some elements within the Israeli government - not the government, some elements within the government - were thinking that by strengthening Mujamma they could put some more pressure on Fatah in the Gaza Strip, back in the mid eighties.

I think it was a mistake, yes."

One of the key factors in Mujamma's rise was the decision by the Israelis in 1978 to grant the organisation official status. This was something that would never have been granted to secular groups. Milton-Edwards says that this was on the orders of the office of the Prime Minister - Menachem Begin, and that former Israeli officials concede that it was part of a strategy to undermine the PLO, divide secular nationalists - and encourage them to join this more conservative alternative.

The former president of the Islamic University of Gaza says:

"They were given permission from the Israeli officials to form. The Israeli authorities kept their eyes closed to the reality of what they were allowing to be created, to the preaching of Islam that was spreading all over the Gaza strip, because at that time the PLO factions had power - and the Israelis wanted an adversary to fight them."

The Israeli military governor of Gaza, Brigadier General Segev even arranged for Sheikh Yassin to be taken to hospital in Tel Aviv to see if the best surgeons in Israel could operate on his spine. They decided they couldn't because they said the damage was too severe.

Bit by bit through the 1980s, with the tacit encouragement of the Israelis, Sheikh Yassin built the structure of an alternative Islamist society in Gaza. All this went unrecorded - I have searched the archives and can find nothing, all the TV reports from Palestine and Israel focus on Yasser Arafat and the PLO. Even when Hamas is formed in 1987 during the first Intifada there is nothing. The first news item about Hamas isn't until December 1992 - when they kidnap an Israeli border guard.

But to give you a sense of the world in which Yassin built Hamas, and of what Yassin is like, I want to show parts of a brilliant film made by the wonderful journalist Sean Langan. He made it in 2001 about the Gaza strip - including going to see Sheikh Yassin at his house. By now Hamas was dominant and its military wing was ordering repeated car bombings of Israeli civilians.

What I love is the way Langan gives you a real sense of the place - both the layout and the mood. It's something that news reports never do. And when he goes to see Sheikh Yassin, Langan's reactions to camera are truthful and honest - scared and silly in equal measure. So much better than the pompous self-confidence of most news reporters which increasingly feels both fake and alien.

But there was a nasty and dark side to what Sheikh Yassin and his fellow Islamists were up to in Gaza in the 1980s. They got a reputation for violently attacking anything that supported the PLO - rather than the Israelis. Milton-Edwards writes:

"After Friday prayers burning torches were held aloft as Mujamma thugs set fire to libraries, newspaper offices, billiard halls and bars. They burned cinemas and cafes, closed liquor stores and ran intimidation campaigns in the community and on the university campus.

Men and women students were severely beaten or had acid thrown at them for speaking out against the Mujamma.

The apparent indifference of the Israeli authorities to such violence was noted by PLO supporters."

An Israeli journalist - Danny Rubinstein - says:

"Ever since, many have accused Israel of providing the raison d'etre for the Islamic religious movement - a phenomenon identical to American support for the Mujahedin in Afghanistan during the Soviet occupation."

But Yassin and the other Gaza islamists did have a sense of humour. One of their main slogans was:

AN UNCOVERED WOMAN AND BEATLE-HAIRED MEN WILL NEVER LIBERATE OUR HOLY PLACES.

And what began to rise up in Gaza was a rigid, limited world view. There is a dramatic expression of this in Sean Langan's film from the Gaza strip. Wandering along the beach he comes upon a group of young Palestinian men - everything goes swimmingly until suddenly they get onto the subject of the Jews and the holocaust.

Suddenly you discover just how much the distorted ghosts from the Nazi era have also risen up to possess the Palestinian mind as well.

When the Intifada began, Sheikh Yassin and other leaders of Mujamma formed Hamas - and Hamas members took part in the ongoing confrontation with the Israeli forces. This was a shift away from the Muslim Brotherhood - who claimed to have renounced violence - but Hamas still saw itself as the Palestinian branch of the Brotherhood.

But Hamas also spent a lot of their time attacking the secular PLO, refusing to have strikes on the same day as the other Palestinian groups, beating up PLO prisoners who were in jail with them - and generally creating divisions within the Palestinian movement. Again the Israelis gave them preferential treatment - not cutting of the flow of funds to Hamas from abroad, and allowing them to keep their schools open. It was all part of a strategy of divide and rule.

At the same time the violence of the Intifada began to create growing divisions within Israeli society. Here are some sections from a fascinating Open Space film made in Israel at the height of the Intifada in 1988. It's made by community activists - and it is following the liberal group Peace Now who are asking for a dialogue with the Palestinians.

But it shows how there was growing opposition to that liberal view. It's actuality footage - with very little commentary - records the moment when you see the progressive optimism of the early Zionism beginning to crumble - and being replaced by a much harsher and above all defensive mood with the rise of the Israeli right. It is epitomised in a woman shouting

"Stinking Arabs - send them all to the gas chambers"

But then Hamas went out of control. The Israelis were worried about its growing strength - and in 1990 they arrested Sheikh Yassin and put him in jail. Their aim was to weaken the command structure of Hamas - but it didn't have that effect at all.

Hamas responed by inventing a "military wing" for themselves which they called the Izz al-Din al-Qassam brigades - after Sheikh Qassam the early Islamist who had fought the British in the 1930s. And in 1992 the Qassam brigades kidnapped an Israeli border guard and threatened to kill him unless Sheikh Yassin was released from jail.

The Israelis refused - so Hamas killed the border guard. There was outrage in Israel - especially from the right who demanded that action be taken against Hamas. The Israeli government went and grabbed 400 of the leading members of Hamas from the Gaza Strip and the West Bank and dumped them on top of a freezing snowy mountain in the south of Lebanon.

It was a public relations disaster for Israel. Day after day news reports showed the Hamas men huddled on top of the mountain. Their organization now became a global brand - and what was worse Hamas attacks on the Israeli forces increased.

It was the beginning of the unstoppable rise of Hamas. here are some of the reports as they unfolded.

At the heart of the Islamist ideas that Hamas was born out of was the belief that secular politicians were dangerous - above all if they used their power to try and change the world.

And in September 1993 Hamas were faced by a secular politician trying to do just that. Yasser Arafat stood on the White House lawn and signed what were called the Oslo Accords - they were agreements that were supposed to lead to peace between the Palestinians and Israel - and a Palestinian state.

Hamas hated it - as also did many from the secular left. They thought that Arafat was selling out the Palestinian people, that the dream of a real liberation had been reduced, as one Hamas leader said, to the dream that Palestinian policemen will have the power to direct traffic.

But Hamas' response would lead them yet again into a very strange relationship with forces in Israel - in particular with the Israeli right who also hated and distrusted the peace process.

Hamas's problem was that many palestinians welcomed the idea of peace - and the promise of safety and calm it promised. But on the 25th February 1994 their chance came to change things. A right-wing Israeli extremist opened fire on Palestinian civilians in a mosque in Hebron. 29 were killed and 125 injured. Hamas promised revenge.

Forty days later - the traditional time of mourning - a Hamas suicide bomber blew up a car bomb in an Israeli town called Afula, killing eight people and wounding many more. Hamas had chosen the town specifically. It had been founded back in 1925 by The American Zion Commonwealth who were an American company set up to try and build model utopian communities that would make the Zionist dream come true.

Afula had been one of these utopian models - built on land bought off an absentee Palestinian landowner. Now it's heart was torn out by a suicide bomber - and it shocked Israel. Hamas was now exploding suicide bombs in Israel with the deliberate aim of killing Israeli civilians. And they followed it up with more - including one in the heart of Tel Aviv.

Here are the reports of the Afula and Tel Aviv bombs. They show the shock and fear that was now gripping Israeli society. And note the politician who turns up at the end of the Tel Aviv report - Benjamin Netanyahu - he says that Rabin's concessions in the peace process have led to this.

Hamas insisted that there was a perfect logic behind the civilian killings - Sheikh Yassin gave interviews saying that if they kill our civilians, then we'll kill theirs. But everyone knew that the real aim was to stop the peace process - to undermine the negotiations between Arafat and Israel.

Then in 1996 there were elections in Israel. The Prime Minister was Shimon Peres who was a veteran of the left-wing Labour Zionist movement. His opponent was the star of the newly rising right in Israel - the leader of the Likud party, Benjamin Netanyahu. He was an opponent of the peace process.

Hamas intensified their suicide bombing campaign. They claimed it was in response for the killing of their best bomb maker - called The Engineer. But in March 1996 Palestinian TV broadcast an interview with a jailed Hamas member who had been organising the bombings.

He was called Abu Warda - and he claimed in the interview that the leaders of Hamas' military wing had told him that the aim of the bombings was to make sure that Peres was defeated, and Netanyahu was elected.

"They thought that the military operations would work to the benefit of the Likud and against the left. They wanted to destroy the political process, and they thought that, if the right succeeded, the political process would stop."

Everyone was furious and all sides - Likud, Fatah, and Hamas said that Abu Warda had been forced to lie. And each blamed the other for doing it. But Netanyahu then went on to win the election by a narrow margin - and he started to do everything he could to drag his feet on the peace process.

And since then Hamas and the Israeli right have been locked together in a terrible cycle in which both shift back and forth between politics and violence in order to promote their aims. Last week's flare up in the Gaza strip was just another example of that cycle.

And at their heart those aims are deeply conservative. Both Hamas and the Israeli right are rooted in defensive ideologies that distrust change and are seeped in a deep pessimism about the ability of politics and politicians to change the world for the better. To try and prevent change both groups have increasingly turned to violence to stop things from running away from them. But it is growing increasingly desperate - because it is impossible to stop the world from changing and the growing addiction to using violence to stop change has corrupted both sides ideals.

But it cannot last. In Egypt, the new President - Mohammed Morsi was elected as a leading member of the Muslim Brotherhood. Yet this week he started acting in the very way the Islamists fear most. He used his political position to ride roughshod over democracy - grabbing power for himself.

In the 1950s Nasser used his power to try and enforce his vision of a progressive, planned world. Now Morsi is doing the same - to try and enforce his vision of a deeply conservative, rigid world.

Again it will fail because it is impossible to control the world in that way - either for progressive or conservative aims. What is badly needed in the Middle East - and in the West - is a new, sophisticated politics that accepts the dynamic forces of history, yet tries to seize them and use the chaotic events of this incredibly exciting time we are living through to try and change the world for the better.

For the last year or so I have been collecting all sorts of footage of people dancing that I found in the BBC archives. In all I gathered over two thousand shots culled from all kinds of programmes. I then cut some of them together to music by the wonderful 70s German band Neu.

I think it gives a sense that we are all together in the dance.

I then took exactly the same sequence of images - I haven't altered even a frame - and put them to a montage of some very different music. There are all sorts of songs and pieces in there - but it owes a great deal to the great romantic musical genius of our age - Burial.

I think that this other version nbsp;you tolook at the people dancingin a very different way. The feeling it evokes ishow separate we are - and how isolated we sometimes are from one another.

Things come and go in the news cycle like waves of fever. A year ago Colonel Gaddafi was killed and an avalanche of camera phone footage of his last minutes was played again and again on the news channels. Then it stopped - and Gaddafi disappeared off into the dark.

What remains is all the footage recording Gaddafi's forty year career as a global weirdo. But the closer you look at the footage and what lies behind it - you begin to discover an odd story that casts a rather unflattering light on many of the elites in both the British and American establishments.

Because over those forty years all sorts of people from the west got mixed up with Gaddafi. Some were simply after his money and they flattered and crept to him because they wanted to be his friend. But for many others he was more useful as an enemy and they helped to turn Gaddafi into a two-dimensional cartoon-like global villain.

Those involved were not just politicians, but journalists, spies from the CIA and MI6, members of Washington think tanks, academics, PR firms, philosophers of humanitarian intervention, posh left wing revolutionaries and the leaders of the IRA.

They all had different aims, and were trying to use Gaddafi in different ways. But underlying almost all of them was a common fear - a feeling that power and influence was slipping away from them, and that increasingly they didn't understand what was going on in the world.

In response, all these different groups began to simplify the world. They all did this in their own ways, but whether they were politicians or journalists or spies, they all began to create an almost pantomime-like picture of the world that maintained their own illusion of control and helped to disguise their loss of power from the general public.

And Colonel Gaddafi happily played a starring role in that pantomime as an absurd clown because it too gave him the global power and influence that he craved.

Together the western elites and Gaddafi helped to lead us into a simplistic two-dimensional vision of the world - full of exaggerations and falsehoods. A fake bubble of certainty that has imprisoned us in the west - and is now preventing us from understanding what is really going on in the world outside.

CHAPTER ONE - THE WILDERNESS YEARS

The story begins back in the mid-1970s with a lonely and frustrated Colonel Gaddafi. He had come to power in 1969 with a burning ambition to transform the world - by liberating the Arab countries from the domination of the west, especially from Britain and America. But no-one would help him - or even cared. He was simply ignored.

Gaddafi was following the vision that had been set out by his hero Gamal Abdel Nasser, the President of Egypt. Nasser had promised to unify the Arab world and transform it into a new revolutionary force that would be strong enough to stand up to the western powers.

But then, in 1970, Nasser died and his vision faltered. Gaddafi tried to keep it alive by unifying Libya with other countries - first with Egypt under Sadat, then with Tunisia and Algeria in something he called "The Steadfastness Front". He even tried Idi Amin in Uganda. But one by one the Arab countries gave up and slipped back to being the compliant puppets of America or Russia.

And Gaddafi was left all alone without any friends.

Here is some of the earliest footage of Gaddafi. It starts with his first appearance ever before the western press.

But Gaddafi wasn't going to give up. He was determined to challenge the old colonial powers.

He was convinced that Northern Ireland was very like Libya. The Catholics, he believed, were fighting a revolutionary struggle against the yoke of British imperialism. So he offered to supply them with money and arms.

He also offered the IRA semi-ambassadorial status - and an IRA supporter went out to live in Tripoli as the "ambassador". His Libyan handlers gave him the code-name "Mister Eddie" and Eddie lived a life of luxury in a grand mansion in the heart of the city eating his meals off the crockery of the deposed King Idris, while old trawlers took lots of guns and semtex from Libya to deserted coves on the coast of Ireland.

Gaddafi also wanted to undermine the west's support of Israel. He supported Palestinian groups fighting the Israelis. But he also decided to do something more dramatic - to send a submarine to torpedo the QE2 that was taking a group of British tourists to visit Israel. Gaddafi mentioned this to President Sadat - who told him that he was completely mad.

And Gaddafi also funded a left-wing revolutionary party in Britain. It was called the Workers Revolutionary Party and its most famous members were the actress Vanessa Redgrave and her brother Corin. The only problem was that it was probably the most useless of all the revolutionary parties in Britain.

It was run a a paranoid Trotskyite called Gerry Healy who believed all other Trotskyites were really CIA double agents. And Healy was also secretly forcing lots of young female comrades to have sex with him "for the sake of the revolution".

I have found a wonderful film that was transmitted just once in a general election programme in 1974 at 4am in the morning. It shows what happened that night when Vanessa Redgrave stood as a WRP candidate for parliament - against a Labour MP called Reg Prentice. It was in Newham in east London and her behaviour as the results are announced shows dramatically why Colonel Gaddafi was backing the wrong revolutionary horse.

It is also very funny and very sad at the same time.

But there was also a sinister side to this relationship with Colonel Gaddafi. Later in the 1980s the WRP held an inquiry into what really went on. The report is still kept secret, but parts of it have been published. If these bits are true, they say that in April 1976 Corin Redgrave had signed a secret deal with the Libyan government for:

"providing intelligence information on the 'activities, names and positions held in finance, politics, business, the communications media and elsewhere' by 'Zionists'. It has strongly anti-Semitic undertones, as no distinction is made between Jews and Zionists"

In other words Corin Redgrave agreed to use the party as Colonel Gaddafi's spy agency in Britain and feed him information about prominent Jews in British society.

Here is Mr Redgrave preparing to do a party political broadcast on the BBC - promising to abolish parliament and create a workers state. But he seems to be most interested in how his tie looks.

By the late 70s Gaddafi decided that there was only one solution to his dilemma. If all the other revolutionaries were so useless - he would have to develop his own global revolutionary theory.

So he did just that and he gave it a name. He called it "The Third International Theory". Gaddafi had discovered what he said was a Third Way, an alternative to capitalism and communism.

Traditional democracy as practiced in Britain and America was a sham he said. It was actually a form of dictatorship. All a party needed was 51% of the vote and it could then impose its ideas on everyone for four or five years - just like clans in Libya did.

The alternative was a new kind of direct democracy in which the people governed themselves. There were no parties - instead Peoples' Committees elected People's Congresses that would manage things. Then there were Revolutionary Committees that made sure the Congresses and their administrators did things in a revolutionary way.

In reality it was a one-man show. Gaddafi made decisions about everything and played all the different committees and congresses off against each other to maintain his power.

But Gaddafi was terribly proud of it. He wrote it all down in what he called The Green Book which he then published in lots of languages because he believed it was a universal, global theory.

You can see just how much this idea pervaded Libyan society from these odd shots I found in some news rushes. They were filmed on a Libyan Ferry going from Malta to Tripoli in the early 1990s. Below decks there are permanent metal signs everywhere explaining the Third International Theory of direct democracy. Good music on the PA system as well.

Gaddafi wanted to tell the world about his vision. He began to invite the BBC to come to Libya and film long interviews so he could explain how important it was.

The trouble though was that every time the BBC interviewers turned up they weren't really that interested in his theory. Instead they wanted to ask him whether he is sending arms to the IRA, and whether he was really planning to torpedo the QE2.

That's what they are really interested in. Not the Third Revolutionary Theory.

Colonel Gaddafi starts off being grumpy about this. But then you can see his face change as he begins to realise what the submarine story is doing for him. That maybe he doesn't need friends - what he really needs are powerful enemies that will make him, and his Third International Theory, infamous, and thus famous.

There is also a fascinating moment in one when Gaddafi breaks into English and describes his time when he came to study in Britain as a young military student. He tells how he went to Beaconsfield and was bullied by British students there. You begin to feel sympathy with him - and then he blows it. They must have been Jews, he says.

CHAPTER TWO - LOONEY TUNES AND CAPED CRUSADERS

Then, in the early 1980s Gaddafi got what he wanted, a global infamy that would make him a powerful presence on the world stage. And he got it because he suddenly became useful to two groups at the heart of the power structure of the West who were facing the growing uncertainty of the time.

One was a new wave of right wing ideologues around President Reagan in America who wanted to find a way of regenerating the moral purpose of their country in the world.

The other was the secret services in both America and Britain. The spies were beginning to realise that the Soviet Union might no longer be a serious threat - and that might threaten their own existence.

What they needed was a new enemy. And the more terrifying, unpredictable and mad the better.

At the beginning of 1981 President Ronald Reagan promised to regenerate America's moral mission in the world - above all to confront the evil empire of the Soviet Union.

But in the back rooms of the CIA, analysts were beginning to question whether this was necessary. They said that all the data they were gathering showed that the Soviet Union was in a terrible state. Even the invasion of Afghanistan, they said, was defensive. There was no way that the Russians wanted to take over the world any longer - even if they ever had.

But the new head of the CIA, William Casey, and the new Secretary of State, General Alexander Haig didn't want to hear this. They were convinced that America had to have something to fight for.

And bit by bit, through the spring and summer of 1981 a new enemy started to emerge in the American newspapers. It was Colonel Gaddafi. State Department officials and other administration "sources" briefed journalists that Gaddafi was at the heart of "the new global disease of terrorism".

In August, American jets patrolling off the coast of Libya shot down two Libyan fighters over the Gulf of Sidra. Gaddafi was furious and began issuing all sorts of threats against America.

Then, in October, a famous journalist called Jack Anderson wrote a sensational article. It said that Colonel Gaddafi had sent a six-man hit team to the US to assassinate President Reagan. Sources in the administration, he said, had concrete evidence that they were led by the most famous terrorist in the world called Carlos "The Jackal".

Then Newsweek said that Gaddafi had equipped them with "bazookas, grenade launchers and even portable SAM-7 missiles capable of bringing down the President's plane". The State department even issued photo fits of the six assassins.

But it seems that it was all completely untrue. Made up by the Reagan administration.

Here are some extracts from a documentary made later in the 1980s in which the journalist Jack Anderson explains how he was fed the story, why he believed it - and how it turned out not to be true.

It also includes an interview with one of the administration men who fed the story to the press. He was part of a committee that had been specifically set up to turn Gaddafi into the mad dog of terror. But even he admits that it was based on very little evidence.

It's a fascinating piece because it is the earliest evidence of what would become known later inside the Reagan administration as "Perception Management". This was the idea that you could use the press and television to tell stories that simplified the world for the American people and turned it into a struggle of good against evil. A cartoon-like picture that justified America's policies in the world.

It didn't matter whether the stories were completely true or not because the overriding moral aim was good.

But Colonel Gaddafi didn't mind the lies at all - because they turned him instantly into a global figure of power and importance.

Gaddafi was a man who understood Perception Management as well, if not better, than the men around Reagan. And he now began to act the part to the hilt. His key ally in this was TV - and in particular the rise of 24 hour news. Underlying it was a shift away from considered packages and towards an exciting sense of immediacy.

Gaddafi was brilliant at it. Here are some of the best bits from the archives of that time. It starts with him appearing on a live satellite link to a mass meeting of The Nation of Islam in Chicago. Gaddafi offers to fund and arm a 100,000 strong black army in America so they can then go and shoot the whites who have oppressed them for so long.

And here is a bit from a film exposing how Colonel Gaddafi has invited German rocket scientists, some of whom had Nazi pasts, to come and build a rocket in Libya. Gaddafi appears in the film explaining that Libya wants to investigate outer space for peaceful purposes. But the film says it may be also so he can attack anywhere in Europe within minutes.

The German company was called OTRAG, and it had previously been building rockets in Zaire for President Mobutu. The BBC had reported on this the year before.

The film has a great graphic showing how the rockets could hit Israel and even Europe. It is remarkably like all the graphics produced about Saddam's rockets attacking Europe in the "dodgy dossier" in 2003

And through it all Gaddafi plays the coy innocent beautifully. He knows just what he is up to.

And here's a great rant attacking America

And then suddenly in the midst of being the world's most dangerous dictator, Gaddafi goes all soft and offers the hand of friendship. He invites the British national team to come and take part in the Libyan International Show Jumping Contest.

The BBC programme Nationwide made a film following the team and what happened. It is just a wonderful film. Not least because it includes a song Gaddafi commissioned - sung in English - to promote his Third Way theory. Plus lots of horses.

Here it is

Then Gaddafi went bad again. He arbitrarily arrested six British oil workers. Lots of people turned up to see him in his tent and plead for their release including a very odd Labour MP with a large plaster on his nose, and the Archbishop of Canterbury's special envoy Terry Waite. There is a good bit where the Archbishop of Canterbury shows off the copy of the Koran that Colonel Gaddafi has sent him.

And the fashion choice Gaddafi makes when he walks in to be interviewed by the BBC's Kate Adie is fantastic. As is his eye-rolling.

But then all this went horribly wrong when WPC Yvonne Fletcher was shot dead in April 1984 by a "revolutionary student" inside the Libyan Peoples' Bureau in London.

Here are some bits of the avalanche of news coverage. Even in the face of the tragic killing Gaddafi plays the cartoon villain - claiming that really it was the British who shot her. But there is also a strange two-dimensionality to the presenters in the studios - and to some of the police involved, especially the detective inspector in charge that day outside the Bureau. His description of how he sees the Libyans as strange aliens sort of sums it up.

At this distance you can see how terrorism and the beginning of rolling news coverage in the 1980s were somehow starting to work together to create a strange construction of an unpredictable but yet simplified world. Figures like Gaddafi, and later Saddam Hussein, along with presenters in TV studios holding up AK 47s, and "opposition" figures shot in shadow all combined with each other to create a weird pantomime version of the world outside.

Perception Management.

This all culminated in 1986 when the Americans bombed Libya, claiming that Gaddafi had been the mastermind behind a wave of terrorist attacks at European airports and the bombing of a discotheque in Berlin.

Reagan explained that Gaddafi was one of the central figures of global terrorism. Along with Iran and North Korea he was part of a set of:

"outlaw states run by the strangest collection of misfits, looney tunes and squalid criminals since the advent of the Third Reich."

And yet again Gaddafi played up beautifully. Here is a fantastic hand held video of him in his bombed-out house the next morning. He calls Thatcher a "harlot" and says that the Americans are trying to stop him spreading his Third Way theory to the young of the world.

And he says that the Americans killed his adopted daughter in the raid - which later turned out not to be true.

But yet again most of the American allegations turned out not to be true.

A year later the BBC journalist Tom Bower made a film that examined the claims in detail. It makes a very powerful case that Gaddafi had nothing to do with the airport attacks. It also looks at the facts behind the Berlin discotheque bombing and questions how much Libya was really behind that as well.

The film interviews men from European intelligence agencies, from Israeli intelligence and even the ex-prime minister of Israel, Yitzhak Rabin. All of them say that the Americans had taken Colonel Gaddafi's mad rantings after 1981 and assembled the fragments of rants and quotes into a dossier that they said was "evidence" of him being a terrorist mastermind.

The hard evidence, they all insist, is that Syria was behind the attacks. They say that Libya had been chosen as a "soft target". It was too dangerous to confront the real culprit - Assad and the terrorist groups he directed - because of the dangers of destabilising the region. Instead you went for Gaddafi - a man without friends or allies. Even the Russians didn't care about him.

CHAPTER THREE - OUR NEW BEST ENEMY

Colonel Gaddafi had willingly helped the west turn him into a pantomime villain. That invented character was then very attractive to those in power in the west because it helped in turn make their simplified, and often fictional, version of the world seem real.

And it wasn't just politicians and spies who got involved in this strange two-way collaboration. Increasingly journalists also found themselves seduced by the special power that Gaddafi had - he could help you transform the world into an entertaining story of global super-villains and a battle against dark forces - and he made it feel real.

But just as had happened with the politicians and spies this would lead some newspapers, and their grand traditions of investigation and truth-telling, to lose touch with reality and create a semi-fictional world.

It began with Arthur Scargill, the leader of the National Union of Mine Workers.

Back in 1984, at the height of the miners' strike, the Sunday Times published a "sensational expose". It said that the Chief Executive Officer of the NUM had gone to Libya, met with Colonel Gaddafi, and that Gaddafi had secretly given money to help the British miners.

The Sunday Times said that the NUM man - Roger Windsor - had travelled there with "the European representative of a Libyan-backed terrorist group" and a high-up man in Libyan intelligence. The terrorist representative apparently also ran a grocers shop in Doncaster.

Here is the TV report that night - including footage of the NUM man meeting Colonel Gaddafi that had been released by the Libyans.

A few months later the strike collapsed, and the story was forgotten. But five years later in 1990 the Daily Mirror and ITV's "Cook Report" programme brought it back to life in a sensational way.

They alleged that Scargill had used Colonel Gaddafi's money corruptly to pay off the mortgage on his house while his members starved. This effectively destroyed Scargill - because although many people thought him vain, pompous and scheming - no one had thought that he was corrupt.

It was full of breathless detail, of the money being smuggled through Heathrow in suitcases, being hidden in biscuit tins and then counted out and distributed in Arthur Scargill's office.

Here is a taste of the avalanche of reaction. Including the Mirror's new proprietor Robert Maxwell challenging Scargill to sue. I have also included an extraordinary shot from some news rushes of a camera constantly pursuing Scargill - up stairs, through corridors, into a ballroom and then through a car park. In its odd way it gives a very good sense of the mood, and of what Scargill was like as a person.

And, as he is followed, notice that Scargill stops to buy a left-wing newspaper called News Line. It is the paper of the Workers Revolutionary Party which had got funding from Colonel Gaddafi.

But it seems that all the allegations of corruption were completely untrue.

It was true that Mr Windsor went to Libya and met Colonel Gaddafi. There was some money that was lodged in a bank account in Doncaster - but none of it seems to have ever got to the NUM or Arthur Scargill.

But more than that - many journalists and MPs who have looked into the whole episode are convinced that Arthur Scargill and the NUM were somehow set up, possibly by MI5. That the trip by Mr Windsor to Libya and the money he said received was stage-managed or manipulated in some way by the British intelligence services to smear Scargill.

On the other hand none of them have produced solid evidence. Some have alleged that Mr Windsor was really working for MI5, which he firmly denies. Others have asked whether the so called terrorist from Doncaster was a plant. He too firmly denies any such thing.

Then - in 2002 - the Mirror editor who had published the original expose, Roy Greenslade, wrote an article saying that he now believed that everything they wrote was false, and that the Daily Mirror with its great tradition of investigative journalism had been duped. It is a very powerful and courageous piece and it ends like this:

"I am now convinced that Scargill did not misuse strike funds and that the union didn't get money from Libya. I also concede that, given the supposed wealth of Maxwell's Mirror and the state of NUM finances, it was understandable that Scargill didn't sue.

Nothing I have said should be taken as criticism of the Mirror journalists: we were all taken in. I can't undo what has been done, but I am pleased to offer the sincerest of apologies to Heathfield and to Scargill, who is on the verge of retirement. I regret ever publishing that story. And that is the honest truth."

You can find the whole article here. It is really good.

In the article Greenslade speculates whether the Daily Mirror had been duped as part of an MI5 plot to discredit Scargill. But he says it remains a strange mystery.

His article though is fascinating because of the wider picture it gives of what was beginning to happen to investigative journalism as it got involved in this cartoon-like world of "internal subversion" and "international terrorists" and mad dictators. It didn't seem to be so much about just revealing the truth any more - rather it was helping create a sense of dark shadowy plots and impenetrable mysteries surrounding modern life.

And it was going to get a lot weirder - and yet again Colonel Gaddafi was going to be at the heart of it

On December the 21st 1988 a Pan Am flight from London to New York was blown up - and the debris came down on the small Scottish town of Lockerbie. It was one of the first of the modern terror panics - and what made it feel more intense and frightening was the avalanche of reporting in the new 24-hours news cycle.

In the face of this, investigative journalism was going to go beyond the fog of immediacy and cut through and tell the truth - what really happened.

And it did - or so it seemed. Within months of the attack the famous Sunday Times "Insight" team had a series of scoops that revealed that the bombing on the Pan Am plane was a revenge attack by Iran for the shooting down of an Iranian airliner by an American warship in the Gulf in 1988. The articles laid it all out in enormous detail - how the Iranians had paid a Palestinian terrorist group based in Syria to plant the bomb in a Toshiba cassette player. And that this had been done with the help of the Syrian authorities.

The terrorists were named and "intelligence sources" were quoted with absolute certainty saying that they knew this is what had happened. There was no mention at all of Libya.

But then suddenly in December 1990 there was there was a complete switch.

"Intelligence sources" in America began to tell journalists that they had found evidence that showed that it was Libya who had masterminded the bombing.

Then in June 1991 the British and American governments formally announced that Libya had been behind the bombing. Here is the first TV report, it includes a conservative MP called Teddy Taylor who had been to see Colonel Gaddafi. He raises the question that was going to lie at the heart of this puzzle.

Isn't it a bit odd, he says, that at the very moment in 1990 when Syria became America's ally in the first Gulf War, that America suddenly stopped accusing it of Lockerbie? And at the very same moment America and Britain suddenly find evidence proving it was Libya.

Suddenly the media was deluged with reports that said that the Lockerbie bombing had been carried out by Libya.

And many of the investigative journalists who had previously said that it was definitely Iran also changed their tune as well. Even the journalists who had written the Sunday Times articles saying there was concrete proof it was Iran and Syria now said it was the mad dog of terrorism - Colonel Gaddafi. And what's more their "intelligence sources" were absolutely sure too.

But a few old-school investigative journalists held out against this sudden swerve. The main one was Paul Foot from Private Eye. He wrote a devastating pamphlet that tears part the whole American and British case against Libya.

Foot showed that much of it rested on the evidence of one extremely dubious witness called Mr Giaka who claimed to be high up in Libyan intelligence. In fact he was a mechanic in a garage who serviced the vehicles for Libyan intelligence - and he had been paid hundreds of thousands of dollars by the Americans.

Even his CIA handlers were very suspicious of him - and after two years of getting nothing from Mr Giaka they told him they would stop paying him unless he came up with some incriminating evidence for the US Department of Justice. The next day Giaka did just that - describing a samsonite suitcase that was loaded onto a plane in Malta by Libyan intelligence. Something he had forgotten to mention for two years.

Giaka explained:

"When I met with the representatives of the Department of Justice, they are very good investigators, and they can distinguish truth from lies. One way or another, they can obtain what they want."

The other key piece of evidence was a tiny fragment of what the Americans said was a kind of timing device that had been sold only to Libya. It too was only discovered to be important 18 months after the bombing - but yet again Foot shows how dubious the claims were that the Americans made about this tiny fragment.

Foot's pamphlet is a powerful piece of journalism that makes a strong argument that the case against Libya is at best massively flawed and more probably a work of fiction.

But it also shows what was happening to journalism - because Foot argues that the tradition of investigative journalism that the SundayTimes Insight team represented had fundamentally changed. And the reporting of Lockerbie and Gaddafi showed this.

When Rupert Murdoch had bought the Sunday Times in 1983 he had appointed a new editor who disapproved of the Insight column and its traditions. Foot says:

"One casualty was the tradition of independent journalistic investigation. This was replaced in the main by material which posed as "investigative" but which in fact recycled information from safe sources, safest of which were the police and the security and intelligence services."

That reliance on sources in the police would come to have disastrous consequences for News International - as we have recently seen.

But the key point back then in the early 90s is that that growing reliance upon sources in the secret intelligence world happened at the very moment when those sources were themselves becoming hopelessly lost. The Cold War was over and all the old certainties were disappearing and the spooks were floundering around, not really knowing what was going on any more.

This made both the intelligence services and their political masters increasingly prey to those right-wing ideologues who had first emerged around President Reagan and who seemed to believe that you could base policy "on not very hard evidence" in order to manage the world.

And some journalists, desperate for crumbs from the powerful went on blithely publishing what they were told by their sources, no matter how illogical, contradictory or phoney it was.

And everyone moved further into a two dimensional world.

But Colonel Gaddafi did still have some friends in Britain. Yet again another group who were feeling power and influence slipping away from them turned to him for help. This time it was the National Front.

By the late 80s the extreme right in Britain were falling apart. To try and save themselves a new younger generation in the National Front decided that racist xenophobia was not enough and that what was needed was a positive, inspiring model for how to organise society as an alternative to the two-party parliamentary system.

And the model for that, they decided, was Colonel Gaddafi's Green Book and its Third Revolutionary theory.

A National Front delegation went to Libya. They asked for money to fund their new project - but all they got were bulk copies of The Green Book. Undaunted they decided to try and set up a model for this new politics in the London suburb of Isleworth.

Here is a lovely bit from a documentary made about what happened to the NF in the 1980s and 1990s. There is a great grabbed interview with the architect of the scheme - Phil Andrews. He admires Gaddafi because he criticises "traditional corrupt politics".

Phil sets out to create a Gaddafi -style popular democracy in Isleworth. To help in this he donates a copy of The Green Book to the local library.

We went to Isleworth Library to see if anyone had borrowed the Green Book since the film was made. But we found that Gaddafi's book is no longer there. Robert Deighton who works in the library said that it had now become "a hub". To do this they had got rid of all the books that no-one borrowed, so it looks like no-one in Isleworth ever read about Gaddafi's Third Way. And now they never will.

Here is a picture of Robert in his "hub".

By the mid 1990s Colonel Gaddafi was all alone again. The sanctions over Lockerbie isolated his country from the rest of the world and his economy began to fall apart. With it also went his vision of the Third Way.

When he held a lavish parade for the 25th anniversary of the Libyan revolution practically no-one came from other countries. But John Simpson from the BBC turned up and did a very good report.

It's good because he cuts through the fog - and says clearly that Gaddafi is really a showman, he is not a serious threat. And that for the past ten to fifteen years Gaddafi has been used as the easy alternative to confronting the serious threats in the Middle East.

Here is his report.

CHAPTER FOUR - WEAPONS OF MASS DECEPTION

But the fake vision of Gaddafi had by now gone very deep in the western imagination. He was at the centre of an interconnecting web of ludicrous, largely fictional stories. And what was now going to happen was that those stories would begin to a coalesce with other simplified and exaggerated stories about other super-villains around the world. Out of that odd stew would come a grand unified theory that would be one of the central beliefs of our age.

MI6 called it "Global Risks" and it was a vision that we now lived in a terrifying world of mad dictators at the head of rogue states who were teaming up with international terrorists, drug barons and ruthless adventurers offering to sell things like smuggled nuclear weapons to the highest bidder.

The world, this theory said, now had to be seen as one interconnected system that transcended nations and their petty preoccupations. And western elites had a duty to defend the system against this new array of "Global Threats". In short they should become world policemen.

MI6 loved this theory because it gave them something new to do. And the obvious place to start was by getting rid of Colonel Gaddafi.

In 1998 a whistleblower from inside the British intelligence services called David Shayler claimed that in 1996 MI6 had paid an Islamist Jihadi group in Libya to kill Colonel Gaddafi. If true it was not very good for MI6 because it meant that agents of the British government were engaging in a programme of assassinating foreign heads of state.

The Islamist group were called The Libyan Islamist Fighting Group. But, Shayler said, they had bungled the operation and detonated a bomb under the wrong car, killing six innocent people.

Both MI6 and the Labour Government denied it. But Shayler agreed to appear on a special Panorama programme that examined his allegations. The presenter - Mark Urban - concludes that Shayler's claims might be true.

Although Shayler's story about the strange things that were going on inside MI6 might be true, Shayler then rather undermined his credibility by some of his subsequent behaviour. He decided to believe some of the conspiracy theories about Sept 11th - and then started dressing as a woman, giving himself the name Dolores Kane and declaring that the world was going to end in 2012.

Rex Features

But it also made no difference because by now everyone was believing in this vision of a world of hidden threats. And the biggest threat of all were WMDS.

Journalists also tried to turn themselves into World Detectives, trying to expose these new terrifying threats - the Weapons of Mass Destruction that the crazy but infernally cunning dictators were hiding. Their sources in the intelligence agencies told them the WMDs were there. Somewhere.

Here are some sections from a documentary made in 1998 about a search for Colonel Gaddafi's WMDs that both illustrates this perfectly - and then at the end shows the empty fatuity of this quest.

It is made by the journalist John Sweeney. He gets into Libya to make a film about Gaddafi's giant water project, but he has an additional aim which is to see if the Libyan's are hiding WMDs in the giant underground reservoirs

There are fantastic, beautiful shots of this extraordinary project as Sweeney tramps around looking for the hidden threats. He finds nothing yet keeps talking about how the CIA say there is the biggest chemical weapons plant in the world hidden somewhere.

But Sweeney is a very good and honest journalist - he has an ability that is very rare in TV reporting to emotionally judge the truth of a situation - and towards the end he confronts his minder about the WMDs. He does it on audio, but to do this he has to keep the video camera running.

What results is not only a piece of avant-garde film making. But the minder is also very sharp. In just a few sentences off-camera he makes you reflect on how ridiculous and paranoid this western mindset has become. And Sweeney gives him the space to do it.

But Colonel Gaddafi was by now in deep trouble, and he was desperate to get rid of the UN sanctions.

In 1999 - pushed by Nelson Mandela who Gaddafi trusted - he agreed that the two men named by America and Britain as suspects could be put on trial in a special Scottish court in the Netherlands. Gaddafi believed that the lack of any substantial evidence would set them free.

But it didn't. One was acquitted, but Abdelbaset al-Megrahi was convicted and sentenced to life imprisonment in a Scottish prison. Gaddafi protested as did lots of Libyans. But is important to realise so did lots of people in Britain. The Professor of Law at Edinburgh University, Robert Black has said bluntly that:

"It is the most disgraceful miscarriage of justice in Scotland for 100 years.
Every lawyer who has read the judgement says 'this is nonsense'. It is nonsense. It really distresses me."

And Jim Swire, whose daughter Flora died on the plane, doesn't believe that the court got anywhere near the truth about Lockerbie. When the verdict was read out in court Swire fainted.

It would seem that possibly Colonel Gaddafi's distrust of Britain and America might not have been just another of his fantasies.

But even then the Americans refused to lift the sanctions until Libya admitted their guilt. And in 2003 Gaddafi agreed to do that. Or so it appeared.

Gaddafi had decided - as had happened throughout this story - that the only way to get what he wanted was to pretend.

Here are some sections from the rushes of an interview with Gaddafi's son Saif al-Islam. Again and again Saif insists that Libya's admission of guilt was a simply a necessary pretence. A lie that was "the only way to exit from the nightmare of the sanctions". They were being forced "to play by the rules of a game invented by Britain and America".

The interviewer is the BBC producer Guy Smith. He is brilliant at insistently pushing Saif about the terrible cynicism and hypocrisy that underlies such a decision. But Saif is also rather impressive in the way he responds. Not only was there no alternative, he says, but everyone involved - even the families of the victims - have become corrupted by the situation. The families are greedy, he says, constantly asking for more money.

If Saif is right - then the picture he gives is a very dark one, where the lies and exaggerations that started back in 1981 have stretched out to corrupt everyone involved.

But then he might be lying.

It is a really good interview.

But Gaddafi understood this fake world better than anyone else - and he was about to play with it, twisting the deceptions even further. His aim was to find a way of getting back to the centre of the world stage, and finally be accepted by those in power in Britain and America.

He was going to do it by doing what he had always done. He would pretend to be more terrifying and dangerous than he really was.

The key, Gaddafi knew, were weapons of mass destruction. America and Britain had invaded Iraq in the spring of 2003 - and they had justified this by claiming that Saddam Hussein had WMDs. But it turned out that he didn't and it was a disaster, especially for Tony Blair.

So Gaddafi decided to help Blair. He admitted that Libya had indeed been hiding chemical weapons and nuclear research facilities, and he offered to give them up.

For Tony Blair this was a godsend because it allowed him to say that the Iraq invasion was having the desired effect of persuading other "rogue states" to transform themselves. And the BBC allowed Blair to break live into the 10 pm news to announce that Colonel Gaddafi had made an historic decision.

The only thing that no-one mentioned was that Gaddafi didn't really have any dangerous weapons of mass destruction.

He had tried to develop nuclear research in Libya, and had bought lots of centrifuges and other equipment. But it had never got off the ground. The CIA would later be quoted as saying that it was way beyond the ability of Gaddafi's scientists even to assemble the equipment.

And one of the leading WMD experts - Jonathan Tucker from the Monterey Institute -said that the chemical weapons were "quite limited". Libya had made mustard gas but it remained in leaking barrels and hadn't been turned into weapon form. As for the more powerful nerve agents Tucker said Libya had tried to make them but turned out not to have the capabilites or the know-how.

Here are some bit of Tucker's 2009 report - The Rollback of Libya's Chemical Weapons Programme:

"The nuclear program was embryonic….while the biological weapons program was little more than a plan that had made minimal progress.

The Libyan Chemical Weapons program…had involved fewer than a dozen chemists and chemical engineers.

The size of the Libyan Chemical Weapons stockpile turned out to be far smaller than the 100 metric tons that the US intelligence community had estimated. Although the Chemical Weapons research program was still active, the production line had been shut down for more than a decade.

The large-scale production of nerve agents was beyond Libya's technological reach."

As the presenters wait for Blair to appear, Andrew Marr sums up what is happening brilliantly. The story that is central to Blair's "world picture" he says is that the modern global threat is "rogue states" coming together with "WMDs". And Gaddafi has just made that story real.

Although of course Gaddafi was - as usual - happily exaggerating how dangerous he was.

There then followed a tidal wave of creepiness with ministers and commentators lining up to say how "courageous and statesmanlike" Gaddafi had been, and how he had "taken a step towards world peace". Culminating in Blair going to visit Gaddafi in his tent.

Blair, along with many commentators, also predicted that this would result in a new openness in Libyan society. Here is Blair's visit, but I have added an interview from 2009 with an internal dissident in Libya to show that even five years later Libya had not changed.

And in return for this, many western institutions and eminent individuals now happily set out to create a new, alternative, and equally fictional image of Libya. It was no longer the dark realm of international terrorism. It was a "reforming" country joining the modern world. Led by an inspired "modern thinker" - Colonel Gaddafi.

Behind a lot of this was an American PR company called The Monitor Group. They were paid $3 million to conduct a cleansing campaign for Libya's image. The aim, according to an internal memo was to:

"enhance international understanding and appreciation of Libya and the contribution it has made and may continue to make to its region and to the world."

They did this by getting eminent liberal intellectuals and leading academics to come out to Libya and have economic forums where they all agreed that the country could develop into a "unique form of popular capitalism".

One of these intellectuals was the famous prophet of The Third Way who had inspired Tony Blair - Professor Anthony Giddens from the LSE. Giddens was flown out and met Colonel Gaddafi. He wrote proudly of how he discovered that his version of the Third Way was similar to Gaddafi's Third International Theory:

"You usually get about half an hour with a political leader. My conversation lasts for more than three. Gaddafi is relaxed and clearly enjoys intellectual conversation. He likes the term "third way" because his own political philosophy is a version of this idea. He makes many intelligent and perceptive points. I leave enlivened and encouraged."

Very NBF.

Giddens was so encouraged that he went out again and took part in a panel of intellectuals chaired by Sir David Frost - and everyone talked about how "authentic" Colonel Gaddafi's conversion was.

Here they are sitting round a modern table - while Colonel Gaddafi reminds himself of his theories.

With this new image Gaddafi then set off to tour the world as a new leader-cum- philosopher. And everywhere he went the westerners who had once laughed at him and tried to kill him now bowed down before him.

Here are the reports of him visiting the European Commission who were so kind as to have recreated and exact model of the Colonel's tent for him to stay in. Then Gaddafi was invited to the UN. By this time the "conversion" seems to be slipping a bit because he goes and makes a speech where he tears up the UN charter and tells them that swine-flu is man made.

Here are the reports of the visits.

The Gaddafi family now become international D-list celebrities.

His son Saadi went to play professionally for Perugia FC in Italy. It was rumoured that the Libyans had paid Perugia to take Saadi on.

Another son - Hannibal - travelled the European party circuit staying in swank hotels. In 2008 he was arrested in Switzerland for allegedly assaulting two of his servants. Although the charges were dropped two days later, the Libyans threatened to stop trade with Switzerland, cancelled air flights, and Hannibal's father withdrew £3.2 billion from his Swiss account. It has been reported that the Swiss then apologised and paid Hannibal compensation.

His extremely glamorous daughter Ayesha was a lawyer. But the "conversion" didn't seem to work very well in her case. She went to Iraq to be one of the defence team in Saddam Hussein's trial.

Here is the fantastic sofa Ayesha relaxed on at her Libyan villa. The photo was taken after the revolution.

But the most sought after was Gaddafi's second son, Saif al Islam, because he was rumoured to be his father's successor.

He too wanted to become a "modern thinker' like his father, so he applied to the London School of Economics. One of the Professors discovered that he was helped in his application by British Aerospace.

Some of his teachers were a bit baffled by their new student. One professor later said - "I could never get clear exactly what he was arguing." But another LSE professor had no such doubts. He was called David Held, and he was a great enthusiast for the idea of "globalisation". And Saif's thesis was very much on message - it was called:

"THE ROLE OF CIVIL SOCIETY IN THE DEMOCRATIZATION OF GLOBAL GOVERNANCE INSTITUTIONS: FROM 'SOFT POWER' TO COLLECTIVE DECISION MAKING"

When someone at the LSE explained Giddens' Third Way theory to Saif, apparently his immediate reaction was "my father invented that thirty years ago."

There was only one problem with Saif's thesis though. It appeared that he might not have written all of it himself. An investigation carried out after the Libyan revolution discovered that Saif had quite a lot of help from the Research Department of the Monitor Group. The same PR company that was flying all the global intellectuals out to meet Saif's father.

Saif probably needed this help because he was busy in other areas. At the same time he was becoming an international artist. He had a travelling exhibition of his own paintings. It was called "The Desert Is Not Silent".

Here are some of the paintings.

At the end of 2008 Saif was awarded his PhD. A few weeks later Professor David Held suggested to him that he might help fund the LSE's Centre For Global Governance. Held says that there was no connection between the two.

Saif said that he would give the Centre £1.5 million, and it would come, he said from his own Foundation in Libya.

What then happened was laid out last year in the investigation written by Lord Woolf. It is a brilliant piece of journalism - and it is a savage expose of what really went at the heart of one of the most eminent academic institutions in the world. The report is really worth reading and it is beautifully written.

A group in the LSE led by David Held were happy to accept the money. But then a Professor called Fred Halliday who had spent his life studying the Middle East rather than worrying about Global Governance pointed something very awkward out. He said that the money that was being offered was dirty money. It was actually bribes paid by western companies in order to secure contracts in Libya.

The LSE investigated and found out that this was probably true - and what were called "due diligence" documents laid this out. But then a strange thing happened. At an LSE council meeting these documents were not presented. Instead David Held described how he had monitored the blogosphere for reactions to the proposal and that there had been no negative comment about the relationship.

Lord Woolf is scathing in a very English way about this:

"I am unable to establish why the due diligence documents did not reach Council. It would have been much more valuable for the Council to have had documents relating to the source of the money rather than media clippings showing perceptions of the LSE's engagement with Libya."

It would seem that the practice of "Perception Management" might now have reached the mild liberal academics.

The loan was agreed, David Held joined the board of Saif's Foundation - and then Saif was asked to give a lecture in the big auditorium at the heart of the LSE.

Here is David Held introducing Saif al Islam Gaddafi's speech. Saif still tries to defend his father's idea that Libya is a better form of democracy than the democracies of the west. He points out how the increasingly low turnout of voters in America has allowed politics to be co-opted by special interests. But then he can't help collapsing into laughter when he says that this means that Libya is a truer form of democracy.

And even Colonel Gaddafi's oldest enemies now became his new best friends.

Twenty years before officers in MI6 had allegedly tried to assassinate Gaddafi by paying a group of Islamist rebels called The Libyan Islamist Fighting Group.

Now MI6 wanted to do everything they could to please their new best friend, Colonel Gaddafi. So when Libyan intelligence asked MI6 help them capture one of the leaders of the Libyan Islamist Fighting Group it seems that they were only too happy to oblige.

One of the leaders of the Fighting Group, Abdel Hakim Belhaj, has alleged that in 2004 Libyan Intelligence asked MI6 to help kidnap him. MI6 and the CIA then traced him and his wife to Thailand where the CIA kidnapped both of them and he was tortured. They were then flown to a Libyan jail and mistreated he says.

Belhaj says that in doing this the British Intelligence agencies effectively were colluding in kidnapping and torture. The evidence of their involvement in his case is strongly backed up by a series of secret memos found by Human Rights Watch in Libyan Intelligence after the revolution.

Here is a very good news report by Peter Taylor about what Belhaj alleges - and how the secret documents back him up.

The British elites now got everywhere in Libya. Here is an odd moment from a documentary made by Simon Reeve that uncovers a telling detail.

When he gets to the city of Sabha Reeve goes to visit the hut that Colonel Gaddafi lived in when he was a schoolboy. It is now preserved in the middle of a roundabout. Reeve leafs through the visitors book and finds an effusive entry from a British General called Robin Searby.

General Searby was Tony Blair's Defence Co-ordinator for Libya. Documents later revealed that General Searby had helped negotiate a deal that would lead to the SAS training Libyan soldiers in "counter-terrorism".

Searby defended the programme by saying that the Libyans were woefully behind in counter-terrorism tactics:

"It was better to have them inside the tent rather than outside"

He added though that the programme had to be abandoned - because "the Libyans were not up to it".

CHAPTER FIVE - DOWN THE DRAIN

But this dream world of global acceptance wasn't going to last. Gaddafi had managed to redeem himself by manipulating a simplified vision of the world that was divided into goodies and baddies in such a way that he became a goodie. But that simple universe had a remorseless logic to it - and Gaddafi was about to be brought down and destroyed by that logic.

Western elites by now saw much of the world through that goodies and baddies prism, so when the Arab Spring began in 2011 it was simply understood as the uprising of the good people against the bad rulers. Two months later the Libyan people rose up against Gaddafi, and that mindset automatically saw the Libyan people as Goodies.

Which meant that Colonel Gaddafi must be a baddie. So everyone switched sides yet again, just like that. And the last dance began.

Here is a short film about that last dance - along with some of Colonel Gaddafi's friends.

The question at the heart of this whole story is - Who was the ventriloquist? And who was the dummy?

Maybe we were the dummy? By allowing perception management with its simplifications, falsehoods and exaggerations to create a simplified vision of the world - we fell into a fake universe of certainty when really we were just watching a pantomime.

And now as the Arab Spring unfolds and reveals the true chaos and messiness of the real world - above all the horror of what is happening in Syria - we find ourselves completely unable to understand it or even know what to do. So those stories get ignored while we follow others with clearer and more simplified dramas which have what seem to be obvious goodies and baddies - thank god for Iran, North Korea and Jimmy Savile.

He was called George King. He was a London taxi driver who back in 1956 had a strange experience. He was washing the dishes when he heard a voice which said

"Prepare yourself. You are about to become the voice of Interplanetary Parliament"

As a result, George King founded the Aetherius Society - which still exists today. His aim was to spread the messages that he received from what he called The Space People.

In 1959 the BBC made a half hour programme about Mr King, his strange cosmic experiences and his ideas. It is one of the most wonderful, odd and touching films I have ever found in the BBC archives.

Here is Mr King

George King is allowed in the programme to describe what happened to him at length - and the interviewer takes him completely seriously. The result is extraordinary - Mr King tells how the space people have given him a name - they call him "Mental Channel No. 1", how he has met people from Mars, Venus and Saturn and has "telepathic rapport" with them.

The interviewer then asks for proof that he really has met these people. George King says that his mother has proved it. She did this, he says, by being picked up by a UFO and then travelling through space to meet one of the people who regularly communicates with him.

So that proves that - he says.

And they then play the tape recording of his mother describing her interplanetary flight.

And then it gets weirder. The interviewer asks George King to contact - and channel - one of these beings. And he agrees. What then happens is just brilliant.

The film begins, appropriately, with Mars speaking to Earth. It ends - as all good programmes did in those days - with a Jungian consultant psychiatrist assessing George King's claims

At the beginning of this year one of the weirdest characters ever to become involved in the present Afghan war died. He was called Jack Idema and he was a brilliant con-man. For a moment, during the early part of the war, Idema persuaded all the major TV networks and scores of journalists that he was some kind of special forces super-hero who was using all kinds of "black ops" to track down and arrest the terrorists.

In reality, before 2001, Idema had been running a hotel for pets in North Carolina called The Ultimate Pet Resort. He had been in prison for fraud, and had tried to con journalists before about being some kind of super-spy. But September 11th gave him his chance - and he turned up in Kabul dressed like this.

And everyone believed him and his stories. In the process Idema brilliantly exposed the emptiness and fakery of much of the TV and newspaper reporting of the war on terror.

He told the journalists and the TV presenters all kinds of lies and fantasies. He even became the central, heroic figure in a book called The Hunt for Bin Laden.

Then Idema charged journalists fortunes for what he said was an "al qaeda" video of a "a training camp" - where strangely many of the terrorists spoke in english, and allegedly you could hear Idema's voice on the soundtrack. Few of the journalists did anything to really check if any of what he was saying was true.

CBS did a special programme about the tapes fronted by Dan Rather, called "Heart of Darkness". They did check on the tapes - the producers went to some of the new breed of "terror experts" that were spawning after 2001. CBS's press office said that they "showed the tapes to three former British Special Forces officers, who verified the tactics being practiced in the video were consistent with those of Al Qaeda".

The BBC did a report that showed the tapes. And they travelled to the village where they had been recorded - and found an old man who said, yes there had been Arabs there.

But much later a number of journalists did investigate Jack Idema properly - and the consensus now is that the tapes are probably fakes.

Here is the original BBC news report

But then Jack Idema started to believe his own stories. He set up his own militia group that he called Task Force Sabre Seven - and he and his men went and arrested Afghans they were convinced were terrorists. And then he locked them up in his own private prison.

Things got out of hand in June 2004 when Idema arrested the Afghan Supreme Court judge, Maulawi Siddiqullah, because he believed he might be involved with terrorists. The judge later described what it was like in Idema's prison:

"The first night, around midnight, I heard the screams of four people. They then poured very cold water on me. I tried to keep myself from screaming, but coudn't. Then they played loud, strange music. Then they prevented me from going to the bathroom; a terrible situation. I was hooded for twelve days."

In July Afghan police raided Idema's house in Kabul and found what was described as a private torture chamber. Eight hooded men, including the judge, were incarcerated there, and three of them were hanging by their feet from the ceiling, with their heads hooded.

Idema and two others were put on trial - and sentenced to ten years in an Afghan jail. And all the journalists puffed a lot about how persuasive he had been.

Here is Idema during the trial - still trying to persuade the journalists that he is what he said he was. And how he is being set up by dark sinister forces.

But what is also interesting about Jack Idema is that in a strange way he may have been ahead of his time.

Because at the moment that Idema was entering his Afghan prison, a group of very senior US military men, led by a General called David Petraeus, were sitting down in a military staff college in Kansas and beginning to write a study that would completely transform the tactics of the US army in Iraq and in Afghanistan.

What General Petraeus and his team did was to go back into the past and exhume a theory of warfare that had been discredited by the US military who thought it was long buried and forgotten. It was called Counterinsurgency.

And out of that would allegedly come the same kind of arms-length, privatised interrogation and torture methods that Idema was indulging in.

I thought I would tell the history of how Counterinsurgency was invented, why it was discredited in America, and how it returned in 2007 to dominate and brutalise the wars in Iraq and Afghanistan. It is a fascinating and weird story that is far odder than anything Jack Idema could have dreamt up - it involves Mao Zedong, John F Kennedy, French fascists, the attempted assassination of Charles De Gaulle, and strange Potemkin-style villages in Vietnam where women get pregnant for no discernible reason.

The theory of Counterinsurgency also had a terrible logic built into it that repeatedly led, from the 1950s onwards, to horror - torture, assassination and mass killing on a far wider scale than anything Jack Idema ever did in his house in Kabul.

The British military (and their associated wonks) like to think that it was Britain's colonial independence struggles in places like Malaya in the 1950s that gave birth to the idea of Counterinsurgency. But the Petraus team in 2006 thought differently. In the foreword to their study, called "FM 3-24 - Counterinsurgency" they point to an enigmatic and long-forgotten French military officer and thinker as their biggest inspiration. They say:

Of the many books that were influential in the writing of FM 3-24, perhaps none was as important as David Galula's 'Counterinsurgency Warfare: Theory and Practice'.

David Galula is an absolutely fascinating figure.

He turns up everywhere in the second half of the 20th century in the wrong place at the right time - like revolutionary China and the Greek civil war in the 1940s, Indo-China in the early 50s, and above all in the French struggle in Algeria in the late 1950s.

In Algeria Galula conducted radical experiments in what was called "revolutionary warfare" - and in these experiments lie the key to understanding the strange revolutionary roots of the theory of Counterinsurgency - and why it could so easily go wrong and lead to horror.

David Galula was born in 1919 in one of the most important colonies of the French Empire - Tunisia. His family were rich merchants and in the 1930s Galula went to study at the prestigious St Cyr military college in France and rose rapidly.

Then, in 1946, Galula was sent to China as the assistant to the French Charge d'Affairs in Beijing. He arrived in the midst of the civil war being fought between the communists led by Mao Zedong and the Koumintang nationalists. A year later Galula went on a trip by himself into the interior and was captured by the communists and held for a week.

Although he was anti-communist, Galula was fascinated by the way the communists behaved towards the local people because it was different from any other troops he had seen. He began to study their tactics which were based on a theory of revolutionary guerrilla war that had been developed by Mao himself.

What Galula realised was that Mao had invented a completely new idea of how to fight a war. Put simply - there was no conventional army any longer, the new army were the millions of people the insurgents moved among. And there were no conventional victories any longer, victory instead was inside the heads of the millions of individuals that the insurgents lived among. If they could persuade the people to believe in their cause and to help them - then the conventional forces would always be surrounded - and would be defeated no matter how many traditional battles they won.

Mao explained the theory in a famous phrase:

"The guerrilla must move amongst the people as a fish swims in the sea"

Here is a picture of David Galula

Galula became convinced that if western armies were going to fight against these new revolutionary ideas they were going to have to change radically. And the way to do it, Galula decided, was to behave exactly the same as Mao's revolutionaries - to swim among the people.

Over the next eight years Galula moved around the world observing the bitter wars of liberation being fought in Greece, Malaya and in Indo China - and he saw how the French army was catastrophically defeated by the communist revolutionary army in Vietnam.

And in 1956 he volunteered to go and serve in Algeria where France was fighting a war against the guerrilla army of the National Liberation Front. Galula found that other officers had been thinking along the same lines - and he was allowed to go and set up what was called "An Experimental Operational Zone".

In a book Galula wrote about his Algerian experiment, that was going to become the bible of the Counterinsurgency movement, he said:

"I felt I had learned enough about insurgencies, and I wanted to test certain theories I had formed on counterinsurgency warfare."

Galula took a village that was in the centre of the insurgency and sent his men to live and work there among the population. The aim was to persuade the people of the village to turn away from the insurgents and thus rob them of their power. The way to do this, Galula said, was through psychological tactics - both by making the villagers feel that they would be safer with the French, but also through indoctrination into a new and modern way of thinking about the world.

If his soldiers and civilian advisers could do this, Galula believed, then the villagers would realise that the real way forward to a better life was not through the insurgents and their vicious tactics, but through the European vision of a new, modern democratic community created amid the harsh mountains of Algeria.

It was a highly idealistic vision - and in 1960 the BBC made a documentary about one of these experiments. It was a "protected village" high up in the Aures Mountains. Galula does not appear - but it is the area in which he was working and is clearly modelled on Galula's theories.

The reporter is the brilliant James Mossman. He was deeply involved in reporting the new wars of liberation that were breaking out round the world - and was no natural supporter of the colonial powers. But he portrays the experiment sympathetically:

"How deeply can the officers influence the minds of the young Algerians by these methods? The officers in charge of the new 'protected villages' make no secret of the fact that this is what they are trying to do. What started as a predominantly military-security operation has blossomed into a fully-blown social experiment"

But at the end Mossman states bluntly "It's all too late"

Here's the film

But there was another side to this Counterinsurgency theory. If you could persuade the local people to come over to your side - then that would leave the insurgents who lived among the people drastically weakened. And that meant you could destroy them.

But to do that you had to identify the insurgents - and that meant getting information from your new "friends" the local villagers. But sometimes they didn't want to give that kind of information, possibly because they were frightened, or they might even be an insurgent themselves, just pretending to be a villager.

And that led to the French soldiers finding ways to persuade the villagers to tell them who was an insurgent. It was called torture.

Here is part of a Panorama film made only two years later in the same Aures Mountain region that revealed some of the horror that had been going on in other of the "protected villages"

By 1962 the French President, de Gaulle had given Algeria its independence. The victorious FLN took power - and it's guerrilla army, the ALN, came out of the shadows, with a great slogan:

Independence is Only a Step. Revolution is our Goal

The Panorama film is a very weird piece of journalism. It treats the ALN like conquering heroes - and slaps an extraordinary piece of romantic music all over shots of them.

But it then shows what it says is the reality of the protected camps and villages that the local population had been put in for "re-education". As the commentary says, the reality was very different from that shown by the French to TV and newspaper correspondents while the war was on.

The film alleges that torture was used in the camps - and then it shows the revolutionaries unblocking an old well outside one of the villages and sending a young boy into the well to find out what is hidden down there.

Here is the section of the film.

There was a terrible paradox at the heart of the theory of Counterinsurgency. As formulated by people like Galula, the theory said that the colonial powers had to imitate the tactics of the revolutionary insurgents they were fighting. They had to become like a mirror that copied Mao Zedong's revolutionary theories - but in reverse, so they could pull the people away from the insurgents.

But what this also meant was that in effect the Europeans became copies of the insurgents - and that could so easily lead them to use some of the same terror tactics as their guerrilla enemy.

The paradox was that while this probably led to less deaths than pointless conventional battles - it also brought torture and murder and a copy of the terrorist mind-set into the heart of the European colonial armies.

This was expressed very powerfully in the film The Battle Of Algiers made in 1966 about the struggle against the insurgents in the Casbah in Algiers. They key figure is the French officer Lt Col Mathieu who comes in to separate out and destroy the FLN insurgents.

This is the scene in the script of the film where he elegantly and rationally argues the case for this logic of Counterinsurgency.

In the French military elite this ruthless extension of counterinsurgency was called "guerre revolutionaire" - or revolutionary war theory. And it captured the imagination of many of the leading officers fighting the war. But it was also going to have very strange consequences for France itself.

When President de Gaulle decided to give independence to Algeria, many of the senior French army officers who had been fighting the insurgents were furious. They believed it was a complete betrayal of everything they had been fighting for, and also of the thousands of French Algerians living in Algeria.

In their anger they set up their own clandestine organisation to try and stop de Gaulle. It was called the Organisation de L'Armee Secrete - the OAS - and among its leaders were some of the officers that had led the Counterinsurgency programme. But Galula himself was not among them.

The OAS was a terrorist organisation that between 1961 and 1962 created an intensely violent campaign of bombings and assassinations in both Algeria and France. At one point they exploded hundreds of bombs a day, killing innocent people, to try and force the FLN to resume their terrorist attacks and thus justify the return of French control.

They also tried to assassinate President de Gaulle five times.

Historians who have studied the OAS have argued that this terror campaign had many of its roots in the "black ops" and techniques of subversion that had been developed by the French military in their Counterinsurgency campaign against the insurgents in the late 1950s - techniques that they had come to believe in deeply.

One historian says that the descent of many of the French officers into...

"..the OAS's terror campaign is inexplicable without their faith in the magical qualities of the counterinsurgency theory"

What had begun as a reverse copy of Mao's revolutionary theories had now mutated into a form of revolutionary terror that was trying to bring down a major European political system.

Here are some of the reports of the time - they give a good sense of the fear and uncertainty that the OAS terror was spreading through France in the early 1960s. Many believed that the terrorism was destroying the very idea of democracy.

I have included an interview with Jean Baptiste Biaggi who was the leader of a fascist group called The Revolutionary Patriotic Party that had risen up out of the crisis. He talks of using revolutionary war to bring down the government - the same "guerre revolutionaire" that had been used in Algiers.

But David Galula had nothing to do with this horrific corruption of Counterinsurgency.

Instead, in the early 1960s, he went to America to spread his idealistic vision of the theory among the US military elite. And the reason he was invited was because an ambitious young Senator had become fascinated with the whole notion of how to fight the spread of communism around the world in a new, revolutionary way. He was John F Kennedy.

Counterinsurgency ideas had first reached America in the form of a best-selling novel. It was called The Ugly American and was published in 1958. It told the story of how communism was spreading through SE Asia - helped on its way by the stupid, arrogant behaviour of the Americans there.

But the hero of the novel - an American engineer called Homer Atkins - behaves differently. He goes and works in local villages to help the local people develop and modernise. Then an American military officer points out that what Atkins is doing is exactly the same as Mao's revolutionary theories set out to do - "win the minds and the hearts" of the local people.

The Ugly American was a runaway bestseller and was later made into a film starring Marlon Brando.

Senator John Kennedy was gripped by The Ugly American. In 1960 he and five other opinion leaders bought a large advertisement in the New York Times saying that they had sent copies of the novel to every US senator because its message was so important.

And on January 18th 1961 - two days before taking office as President - Kennedy set up the new Special Group, Counterinsurgency in the Pentagon - SGCI, led by a powerful General.

But there was only one problem - it couldn't find any real Counterinsurgency experts in America.

So David Galula was invited over by the American military as one of the few people who knew what the new President was on about - and had even written a book about it. And in April 1962 Galula was one of the main guests at a now legendary symposium on Counterinsurgency held by the RAND Corporation military think tank.

All sorts of people were there - like Lt. Col. Frank Kitson who had led the British struggle against insurgents in Malaya, and the mysterious American Colonel Edward Lansdale who was involved in attempts to overthrow Castro in Cuba, and was fascinated by communist theories of revolutionary war.

Galula was the star guest and he got up to speak first. To begin with he put forward his fundamental theory.

"Revolutionary warfare requires a revolutionary approach on both sides in the struggle. Whereas in ordinary war the objective is to destroy the enemy and occupy his territory, the guerrilla's aim is to control the population.

This, therefore, must be the aim of the counter-guerrilla as well"

But then Galula put the boot in to the aspiring counter-insurgents. Whether it was due to his disenchantment with what had happened in Algeria is not clear - but Galula laid out the central problem for the counterinsurgents when they tried to mirror the communist revolutionaries - they didn't have a cause:

"One basic difference between insurgency and counterinsurgency is that the insurgent starts out with nothing but a cause and grows to strength, while counterinsurgent often starts with everything but a cause and gradually declines in strength to the point of weakness"

So the RAND corporation decided to find something equivalent to a cause, powerful enough to bring the villagers in SE Asia over to the American side.

Up to this point RAND had been exclusively dealing with the tactics of nuclear warfare, but in the mid 60s it turned its attention to counterinsurgency - or what they started calling COIN. And very quickly there was a furious debate within the think tank.

The traditionalists argued that you stuck with the Hearts and Minds approach - or what they now called HAM. But others said that this never worked because the Americans didn't have as powerful a vision to offer the peasants as the communist revolutionaries did. They didn't have a romantic picture of creating a new world.

The solution, they said, was to fuse counterinsurgency with modern economic theory - above all the theory of the free market - and treat the villagers as "rational actors" in an economic system. You didn't offer them a vision, or a cause, instead you gave them "selective incentives" to co-operate with the government, plus disincentives to stop them resisting.

One of the men behind this new approach was an economist at RAND called Charles Wolfe Jr. Here he is in 1965 - looking like an economist.

The new theory was called:

"The Cost/Benefit-Coercion theory of Counterinsurgency"

It still believed in Galula's theory of putting the local population into protected villages and making them feel safe. But it gave up on worrying about what was in the villagers heads and treated them instead as self-interested "rational actors" who would respond in more or less predictable ways to incentives - and to disincentives.

It was best summarised in a later book written by another economist called Samuel Popkin all about the cost/benefit calculations of Vietnamese peasants. It was called "The Rational Peasant"

In some ways the shift that happened in Counterinsurgency theory was a picture in microcosm of the much wider shift that was going to happen to all Western societies over the next thirty years. Politicians would give up on the idea that politics was about inspiring the people - and giving them a vision of changing the world. Instead the politicians would adopt the ideas, and the language, of economics, and turn to treating their population as individuals who could simply be incentivised and disincentivised by appealing to their self-interest. You didn't change society any longer - you managed it.

But when this new hybrid theory of counterinsurgency was applied in South Vietnam in the 1960s it didn't work out the way it was intended. To begin with it led to absurdity. And then, just like in Algeria, it led to horror.

Here is part of a film made in 1969 which brilliantly illustrates the absurdity. It is about a South Vietnamese village called Bin Hao - which was held up by the Americans as the model of a pacified village, an example of how their theory and its use of incentives was really working.

But then the Americans discover that the village that their theory has created isn't at all what it appears to be. Their worries begin when many of the women become pregnant - yet there seems to be only old men in the village. And then they discover something much worse.

I have also put at the front of the film a wonderful couple of minutes of two civilian "advisers" in Vietnam playing a board game called "Insurgency". It had been designed by one of the team to express and test out their theories. It sets the weird context for the even stranger reality that then follows.

But, just as in Algeria, the counterinsurgency programme had its own logic that led to torture and murder.

The aim of the protected villages and all the incentives was to separate the population from the insurgents. The next objective was to destroy the insurgents - and to do this the CIA set up what they called The Phoenix Programme.

One of the men in charge was another RAND theorist called Robert Komer. He knew David Galula and had read his books, but he took a rather tougher approach which was summarised by his nick name - "Blowtorch Bob".

Here is a picture of Blowtorch Bob briefing President Johnson on the sort of things he is up to.

What Komer and the others who ran the Phoenix programme did was set up camps to train Vietnamese militias - drawn from the rational peasants. Their job was going to be to root out and kill the communist insurgents. Following the counterinsurgency theory, the militias were direct copies of the communist cadres.

But also - following the economic model - they were to be incentivised. They were given money for killing Vietcong, twenty pounds for a village official, thirty for a district officer.

In a really good documentary made in the 1990s about the Phoenix Programme, Robert Komer appears. He is a great character - and he is absolutely blunt about what his aim was with the militias:

"We would use the Vietcong techniques to beat them. They conducted a terror campaign, so I thought we had to conduct a counter-terror campaign to kill the VC assassins. And we did."

Another of the architects of the Phoenix Programme, who is interviewed, was Nelson Brickham. He was a devotee of David Galula's ideas and he took Galula's books everywhere he went in South Vietnam. He claims in the film to have been "the conceptual father of the Phoenix Programme" - and says that it worked well.

But others involved have now changed their mind. The film shows, with first hand testimony, how that counter-terror campaign ran out of control. Men who directed the campaign for the CIA say that in essence it led them to become terrorists themselves.

Here is that section of the film with Blowtorch Bob and the other Americans telling what happened.

There is a strong counter-argument to these criticisms. It simply says - so what? In war killing happens, and a programme of targeted assassination certainly killed far fewer civilians than the horrific indiscriminate bombing by America's conventional forces.

But the documentary goes on to show how the Phoenix programme created something much worse - which it was powerless both to understand or to stop.

The Rational Peasant approach looked at Vietnam as a society of millions of self-seeking rational individuals. In reality, Vietnamese society was far more complicated. Extended families had tangled and intricate histories of relationships - some were friendly but many were driven by rivalries and hatreds.

As the film makes clear this had created a powerful tradition of violent retribution in Vietnamese society - and it goes on to show how some of the militias that the Americans had created used the free rein their masters gave them, to kill and torture not communists, but other, innocent civilians against whom they had long-standing grudges or hatreds.

One CIA officer describes how he found that the local police chief was using their programme's safe house to torture and carve up people who didn't have the right family protection.

An innocent Vietnamese woman who was tortured describes how the Americans just stood and watched.

It shows the terrible limitations of the economic model of society. The Americans were helpless because their militias would assure them that the people they were torturing were communists.

And when you look at everyone as simply a "rational actor" you have no way of knowing whether they were telling you the truth or not.

After such experiences in Vietnam the whole idea of Counterinsurgency in the American military was discredited. It was buried away and forgotten.

It was replaced in the 1980s by what was called the Powell-Weinberger doctrine. This said that the US should only get involved with a conflict where there are clear objectives and it can use overwhelming force.

But after the invasion of Iraq in 2003 the Americans became bogged down in a new guerrilla war. And so - in 2006 - General David Petraus' team dug up Counterinsurgency again. They took David Galula's ideas and made them the central architecture for a new idea of how to rescue Iraq from the horror that had engulfed it since the invasion of 2003.

At the beginning of 2007 Petraeus was given 20,000 extra troops - and he used them to create a counterinsurgency campaign in Iraq modelled on Galula's theories from the 1950s.

And Galula's central idea - copied from Mao Zedong's revolutionary theory of warfare - that you swim among the people like a fish in water, became the driving idea behind "The Surge".

Here are some unedited rushes recorded in July 2007 - they follow General Petraeus visiting Baqubah -which had been one of the most vicious battlegrounds of the insurgency. General Petraeus shows the BBC reporter John Simpson how the surge is working. As Petraeus talks you can hear the ghost of Galula and his ideas. It is just like the French officer showing a BBC reporter around The Experimental Zone high up in the Aures Mountains fifty years before.

But - just like in Algeria - there were also suspicions about what might be really happening. That the Iraqi army and police were also involved in sectarian killing under the cover of the surge.

John Simpson is a very good reporter - and he knows this and asks the sharp question of the local commander that Petraeus is visiting. The officer's response is, to put it mildly, a bit naive.

And there were other suspicions about the Iraq Surge of 2007. That there was something far more violent and sinister behind it than the simple Hearts and Minds approach.

In his book - The War Within - the reporter Bob Woodward challenges the myth of the surge. He says bluntly:

"The truth is that other factors were as or even more important than the Surge.

Beginning in about May 2006 the US military and intelligence agencies launched a series of TOP SECRET operations that enabled them to locate, target and kill key individuals in extremist groups

The operations, which were part of Special Compartmented Information (SCI) incorporated some of the most highly classified techniques and information in the US government"

Then, rather strangely for an investigative journalist, Woodward becomes very coy. He says:

"Senior military officers and officials at the White House have asked me not to publish the details or the code word names associated with these ground-breaking programs. They argue that the publication of the names alone might lead to the unravelling of state secrets.....

But a number of authoritative sources say these covert activities had a far-reaching effect on the violence and were very possibly the biggest factor in reducing it. Several said that 85-90% of the successful operations and 'actionable intelligence' 'had come from these new sources, methods and operations."

The words 'actionable intelligence' are a bit opaque - but they do imply that there was, just like in Algeria and in Vietnam, a large-scale programme of targeted assassination.

There have also been claims made that, again just like in Vietnam, the Americans gave over much of the operation of the programme to the local militias that they had trained. In this case these were the predominantly Shia Iraqi army and police. And these Iraqis, it is alleged, then hi-jacked the programme and used it to torture and assassinate their Sunni enemies on a wide scale.

There has been little reporting of this, but the Rolling Stone journalist Michael Hastings has just written a brilliant and fascinating book called The Operators. It is about the man who commanded the Joint Special Operations Command in Iraq - General McChrystal. JSOC's job was to run the special units involved in hunting down and assassinating the insurgents, In the book Hastings describes what seems to have happened:

"Behind the scenes, McChrystal, operating his own Phoenix-like Special Ops program, wipes out "thousands," according to McChrystal's deputy, Major General Bill Mayville, noting that "JSOC was a killing machine.".......................

The COINdistas strive to prove the surge strategy is an enlightened form of combat - "graduate level of war" as the manual FM 3-24 calls Counterinsurgency. But the reality on the ground is dark and not very reminiscent of graduate school.

Petraeus and his allies decide to team up with a Shiite Islamist government, picking the majority's side in a civil war. The Americans themselves round up tens of thousands of youg Iraqi males. The Iraqi army and police, fully funded and trained by the US military, conduct a campaign of torture and killing, assassinating suspected enemies and abusing Sunnis with electric shocks and power drills, with entire units being used as death squads. The Sunnis respond in kind.

The American response to this campaign, as the New York Times would later note, was an 'institutional shrug'."

Very much the response of a Rational Peasant.

In June 2007 Jack Idema was suddenly, and rather mysteriously, freed early by President Karzai.

He went off to live in a house in Mexico and ran a charter boat for tourists. He had a new girlfriend called Penny who had corresponded with him while he was in prison in Afghanistan. Then last year he fell ill, and died in January this year of AIDS-related complications.

After his death Penny described how in his in time in Mexico Idema got lost in the different personas he was playing. For the tourists on the charter boat he played the role of what he called "Captain Black Jack". Idema modelled this on the character of Jack Sparrow played by Johnny Depp in Pirates of the Caribbean.

But at home Idema dressed up in Arab Robes - like Lawrence of Arabia - drank heavily, took cocaine and continuously played Arab music, the soundtrack to Apocalypse Now, and What a Wonderful World by Louis Armstrong.

And David Petraeus is tipped by some to be the next US president but one.

But what is hardly ever mentioned in the press and TV reports is that this very system of oppression, the rock against which the dreams of democracy are being dashed, was largely created by the British. That, throughout most of the twentieth century, British advisers to the Bahraini royal family, backed up by British military might, were central figures in the creation of a ruthless system that imprisoned and sometimes tortured any Bahraini citizen who even dared to suggest the idea of democracy.

The same British advisers also worked with the rulers of Bahrain to exercise a cynical technique of divide and rule - setting Shia against Sunni in a very successful attempt to keep Bahrain locked in an old, decaying and corrupt system of tribal and religious rivalries. The deliberate aim was to stop democracy ever emerging.

The Bahrainis know this, practically everyone else in the Arab world knows this - the only people who seem to have forgotten are the British themselves.

So I thought I would tell the story of Britain's involvement in the government and the security of Bahrain over the past 90 years. Especially as the present King of Bahrain is coming to have lunch with the Queen on May 18th.

It began in the summer of 1925 when a young administrative officer in the British Colonial Service called Charles Belgrave read an advertisement in the middle of the "Personal Column" in the Times. It said:

Belgrave answered the mysterious advertisement and was then summoned to an interview in a West End hotel. His interviewer turned out to be one of the heads of the India Office - the government department which ran that part of the British Empire.

What Belgrave was offered was the job of being the British "adviser" to the new ruler of Bahrain, Sheikh Hamad bin Isa Al Khalifa. The precise nature of the job was a bit murky (a murkiness that was going to run through this whole story). On the surface Belgrave would be completely independent of the British government - but what was also clear was that he was being sent there to deal with growing demands for reform and modernization that might threaten Britain's interests.

Ever since 1820 the British had dominated Bahrain. The Al Khalifa family ruled, but in reality it was protectorate whose affairs were "guided" by the British. In 1923 the previous ruler had gone berserk and started terrorising his people - so the British had removed him and installed his son. It was clear to Belgrave what his job was - to create a more centralised form of control in Bahrain and to manage the instability created by the previous ruler's reign of terror.

Belgrave took the job. And here is a picture of him sitting happily in "the Adviserate drawing room"

Belgrave soon became very powerful - and by the 1930s he was in effect running the government of Bahrain. The thing that gave him a supreme ability to manage any dissent was the fact that he ran the courts. Bahrain had no legal code - which allowed Belgrave as judge enormous power. Belgrave described it in his autobiography:

"I found that there was no written code in Bahrain so judgements had to depend on common sense alone. It was rough and ready justice, but it had the advantage of being speedy.

I sat three days a week with a minor Shaikh who was deaf, dull and averse to making decisions. When I asked his opinion he invariably replied, 'I think the same as you Excellency; I agree with whatever you say."

Many Bahrainis soon became convinced that Belgrave was using his power to make sure that the status quo was maintained and to prevent a modern, democratic political system developing. And in the 1950s this anger with Belgrave burst out in a dramatic and violent way - a popular revolt and demands for democracy uncannily like the events unfolding in Bahrain today.

It started in 1953 when a Shia religious parade was stoned and then a Shia neighbourhood attacked by groups of Sunni fanatics. Many believed that it was a deliberate provocation - to create sectarian divisions. People noticed that among the attackers were members of the ruling family including the brother of the Sheikh.

If it was a provocation - then it succeeded. For two years Bahrain was torn by Sunni vs Shia violence. In private Belgrave sympathised with the Shias, but as the public face of the Law in Bahrain he was ruthless. He handed down sentences that were far tougher on Shia rioters than on their Sunni counterparts. And this in turn led to even more rioting.

A group of leading middle-class Bahrainis set up the Higher Executive Committee. It was composed equally of Shias and Sunnis - and it called for Belgrave to go. He was helping foment religious hatred and imprisoning innocent people, they said, in order to keep Bahrain as a tribally controlled regime. They demanded instead democracy and a code of law.

Here is a picture of the Committee.

Things came to a head when in 1956 the British foreign secretary, Selwyn Lloyd, flew to Bahrain for a visit. There was a large, violent demonstration with hundreds of Bahrainis trying to tell Lloyd to remove Belgrave - because he was standing in the way of making Bahrain a modern democracy.

The riots and the demonstration made the news in Britain - and Panorama came out to investigate. The report - by Woodrow Wyatt (later to become one of Rupert Murdoch's closest advisers) - is really good.

Wyatt interviews Belgrave who has a great quote about the demonstration - "it's anti-British, anti-Sheikh, and anti-me." But Wyatt also goes and talks to people on the street, almost all of who want Belgrave to go. One of them standing on the back of a truck sums it up neatly: "Belgrave is not just an adviser - he is the judge, and when he goes to the court he is also the police commandant. He is everything in Bahrain, he is not an adviser."

Faced with this instability the British government moved troops in at the end of 1956 and crushed the revolt. Three of the leading members of the committee were put on a Royal Navy ship and taken and imprisoned on the island of St Helena in the middle of the South Atlantic. The same place that Napoleon had been dumped in 1815. One of them was Abdul Aziz Al Shamlan who is the committee member interviewed in the Panorama film.

This is a picture of their prison, plus a map - the purple blob shows where St Helena is.

But Belgrave had also outlived his usefulness - and the same year he too was dumped by the British (and by Sheikh Khalifa). He came back to Britain and wrote a self-serving autobiography which ends up suggesting that the Arabs aren't "ready" for democracy yet.

And things quietened down in Bahrain.

Until 1965 when another popular uprising began. It began in the oilfields but quickly spread to general strikes. Again the British sent in troops to crush the revolt - and many of the leaders were yet again deported.

But it didn't do much good for the British government - because both press and television in Britain began to ask what exactly was this weird feudal state that we were supporting? And why?

Across the Arab world people had been inspired by the new ideas of Gamal Abdel Nasser, the President of Egypt, and they wanted freedom from the corrupt old Shaikhs and Kings who were propped up by the west. And in 1966 the BBC went out to Bahrain again and made a Panorama programme that tore into the hypocrisy of what Britain was doing in that country.

It didn't pull its punches - the reporter, called John Morgan, says to the camera at the end:

"If one of the tests of a society's health is a citizen's willingness to speak his mind freely in public then Bahrain belongs in the class of a Communist or a Fascist country - and we are deeply implicated in order to preserve our oil and foreign policy."

In the face of this the British government decided the only solution was to find another "adviser". The idea was that on the surface he would appear to be a freelance mercenary who was employed by the ruling Khalifa family. But in reality he would be chosen and placed there by the British Foreign Office to manage the internal security of Bahrain. His job was to prevent the instability that political change would inevitably bring - and the consequent threat to British interests.

The man the British chose was called Colonel Ian Henderson. He had been a colonial police officer in Kenya in the 1950s and had played a major role in suppressing the Mau Mau rebellion. The Kenyans were convinced that Henderson had been involved in ordering both torture and assassination during the rebellion - and the moment the country achieved independence in 1964 its new leaders threw Henderson out.

Here is Henderson being interviewed at Heathrow the day he flew back. I think you can get a very good sense of what he is like - especially in his slightly frightening matter-of-factness. Speculating on the reasons for his expulsion he says, with a faraway look in his eyes:

"What I did many years ago as a police officer during the emergency is today not seen as something very desirable."

Well - yes.

A little while ago a Scottish journalist called Neil Mackay uncovered secret Foreign Office documents that show that the senior British diplomat in Bahrain in 1966 - Antony Parsons - worked on the ruling Sheikh Khalifa to persuade him to appoint Henderson as head of what was called the Special Branch - and to give Henderson a free hand to reorganise it into an efficient, modern covert surveillance "anti terrorist" organisation.

To begin with Henderson presented himself a a new breed of security chief. He freed all the prisoners from the 1965 uprising and announced that the country would now be ruled by proper law, not arbitrary detention. He also persuaded the Khalifas to welcome back militants from protest movements like the Bahrain National Liberation Front and the Popular Revolutionary Movement.

It was all very nice, but many Bahrainis now believe that what Henderson was also doing was building up an intricate system of infiltrators and double agents inside the protest movement - in preparation for the day when Britain pulled out of Bahrain and gave it independence.

That came in 1971 and for a moment ordinary Bahrainis had a modern political system of democracy. In 1973 the ruling Amir - Isa bin Salman al-Khalifa - approved a constitution for the country, and the first parliamentary elections took place.

But then something very sinister happened. Within a year Colonel Ian Henderson proposed a new law he called the State Security Law. It said that any Bahraini could be held for three years without charge or trial on just the suspicion that they might be a threat to the state. It was known as 'the precautionary law'.

It caused an outrage - because it meant that anyone could be imprisoned just on the imaginative suspicions of Colonel Henderson and his State Security acolytes. Parliament rejected the bill in June 1975 and there was a standoff with the regime, and with Henderson.

The Amir solved it in the simplest way - he suspended those articles of the Constitution that guaranteed freedom to the people, and he suspended parliament.

And in August 1975 Henderson went to work. His men began to fill up Bahrain's jails with activists - and among them were members of the now deceased parliament. And for the next twenty five years Henderson ran a ruthless system of repression that kept the al Khalifa family in power and stopped any movement towards democracy.

Opposition activists and human rights groups have repeatedly alleged that this repression has involved widespread torture, the rolling imprisonment without trial of thousands of people, deaths and assassinations. Henderson denies this. In the face of the charges Ian Henderson has repeatedly said that he has never been involved in torture nor has he ever ordered his officers to torture those who have been arrested.

One of the key questions is whether this repression was still in Britain's interest? On the one hand you can argue that it protected the flow of oil, that it kept Bahrain as a bulwark first against communism and then from the 80s onwards against Shia Islamist revolution - plus that Bahrain also became the home to the American Fifth Fleet.

But you can also argue that by inserting Ian Henderson into the Bahraini system of power and security in 1966, the British created an infernal machine that just kept on running after they left in 1971. That machine had been told to prevent any political protests that might destablise the country - and that's what it proceeded to do. The Al Khalifas loved the machine because it kept them in power - and as a result hundreds and thousands of Bahrainis were left stuck with a vicious ghost from the failure of the British empire.

And true to form the British in the 1970s ignored the repression and the torture going on around them. Here are a selection of films the BBC made about Bahrain in the 1970s

First is an extract from a film made about the town of Awali - where all the British oil workers lived. It is an extraordinary place because in the middle of the desert the British have created a copy of a Surrey suburb where they live in blissful separateness from the rest of the country.

Except at the end - when a British couple being interviewed suddenly start describing how strange it is - they say it's like "living in a cotton wool world. I think it is really bad to live here in a world without responsibility. This place steals your life away"

And here are some extracts from one of the oddest arts programmes the BBC has ever shown. It follows a musical composer called David Fanshawe (and collector of Arabian folk music) as he creates his new work called "Arabian Fantasy" in various locations around Bahrain.

He does this by banging oil pipes and machinery in the oilfields, by assembling lots of oil tankers and signalling them with flags to blow their hooters, all interspersed with helicopter shots of him playing his synthesizer in a prog-rock kind of way in all kinds of locations around the island state.

It's made even odder by the appearance of Fanshawe's sidekick who had built his own very complex synthesizer that treats and distorts all the noises. He's called Adrian Wagner - and is a descendant of the famous composer.

Fanshawe is doing all this because he grew up in Bahrain as a boy when his uncle was the naval commander of the British fleet there in the 1950s - and there are bits of him wandering nostalgically round empty expat swimming pools. He's quite annoying - and he seems to like funk music as well.

Then in 1979 the Queen of England came to visit Bahrain - and I've stumbled on the unedited rushes of her visit. Here are some of them. I've listened through to all of her and Prince Philip's overheard conversations with the ruling Amir - and she doesn't seem to mention any of the repression, imprisonment without trial, or killings.

But she does have to suffer a rather strange dance which is apparently expressing how the rights have women have been progressing in Bahrain. At least that's the only thing she had to suffer - unlike many Bahrainis.

And Bahrain had other uses for Britain in the 1970s. In 1979 the BBC made a very creepy documentary film about how Bahrain had become a central hub for the new supersonic jet - Concorde. The truth was that at that time practically no other country wanted Concorde because of the very loud sonic booms it made - and the Al Khalifa family stepped in to save British Airways.

This is a section from the documentary where the British Airways manager Tim Phillips goes to see the Amir at the regular Majlis - where people come to petition and lobby their ruler. Phillips seems to be convinced that the Majlis is almost a better form of democracy than we have in Britain. It is followed by the very creepy scene when he gets to talk one-to-one with to the Amir, and the scene sums up in a nutshell Britain's relationship with this weird state.

Ian Henderson soon became the Dr Evil of Bahrain. He was hated because he was seen as the man whose security law had helped destroy the Constitution and democracy in the country.

In response a new protest movement began to grow which united the secular left and Islamists around the simple, dramatic demand that the constitution and parliament should be restored. It grew slowly at first - but in 1994 it emerged as The Constitutional Movement - and it set out to confront Henderson.

It was the biggest revolt yet seen in Bahrain and it had widespread popular support that crossed across the Shia - Sunni divide. Henderson and his security forces responded viciously. The opposition accused them of using the same tactics of divide and rule that had been seen in the 1950s, deliberately fomenting sectarian hatreds. Henderson's forces were also accused of imprisonment and torture on a scale not seen before.

And - just as in 1956 - at the very heart of the Constitutional Movement's demands was the removal of the British "adviser" who they said was the mastermind behind the terror that was engulfing the country. In the words of the opposition:

"Security and special branch chief General Henderson, along with a bunch of British mercenaries who are in control of the security apparatus bear full responsibility for the deterioration of relations between people and regime and for the festering political crisis - by their policy of sectarian discrimination, by waging large scale arrests and killing campaigns, and by fabricating plots designed to alienate the masses from the movement."

And finally the British noticed. Here is a really good report made for the BBC in 1996 by the brilliant reporter Sue Lloyd Roberts. She uses secret filming and blurred interviews to show what was really going on and evoke the fear that the rolling repression was creating for hundreds of thousands of Bahraini people.

And just like in the 1950s the publicity became too much. In 1999 a new member of the Al Khalifa family took over the leadership of Bahrain - and he decided to finish with Ian Henderson's services.

Henderson returned to Britain where various human rights groups and MPs persuaded the Home Secretary to get the police to investigate whether Henderson could be prosecuted for ordering torture. But the police found that the Bahrain government refused to give them any evidence. So they gave up.

The new Amir also abolished Henderson's hated State Security Law - and announced there would be elections to parliament. At first it all seemed to be a genuine return to the democratic dreams of 1973. But it wasn't. By 2010 it had become clear that the new parliament had practically no real power.

Then came the events in Tunisia at the beginning of 2011 - and it reactivated the opposition in Bahrain. They occupied Pearl Roundabout in Manama - but on the night of the 17th of February the protestors met the full force of the Bahraini security forces.

Ian Henderson might have gone away - but the ferocious system that he helped build hasn't and it haunts the Gulf still today.

It is a lovely film - directed by a brilliant documentary maker called Dick Fontaine, and beautifully shot in the fluid way of cine-verite back then. But more than that it captures the rise of a phenomenon that has come to dominate (and possibly strangle) western metropolitan society today. It is the rise of the hipster. By this I don't mean the present cliche of the ironic moustaches that live in Hoxton and Hoboken - but a new cultural elite that was beginning to emerge at that time, the rebellious, stroppy bohemians who looked to culture rather than politics to define their identity - and above all their difference from others.

In the film you can see them peeking through in the backdrop of many of the scenes, for Mailer attracts them. In his perverse individualism and rebelliousness - one of his slogans is "We're no good, and we can prove it" - Mailer captured a new sensibility. This was because he combined a revulsion against a tired old culture together with a distrust of the political system, and the hipsters loved it.

But Mailer was a complicated man - and as well as embodying many of the hipster values he was also a perceptive and vocal critic of the new sensibility. Back in 1957 he had written an essay for Dissent magazine called The White Negro. In it he had described how fears of nuclear annihilation had begun to produce a new kind of young alienated being in America. These hyper-individualists trusted only their own feelings and desires and refused to be part of any group or organisation. And in black culture, Mailer said, they found their identity - the culture of the dangerous outsider.

This outsider culture had originally been created, Mailer wrote, by blacks in response to racial oppression and violence. But for the "white negroes" that culture was then co-opted in order to give a meaning and grandeur to their psychopathic narcissism:

"In such places as Greenwich Village a menage-a-trois was completed - the bohemian and the 'juvenile delinquent' came face to face with the Negro, and the hipster was a fact in American life. If marijuana was the wedding ring, the child was the language of Hip for its argot gave expression to abstract states of feeling which all could share, at least all who were Hip. And in the wedding of the white and the black it was the Negro who brought the cultural dowry.

So there was a new breed of adventurers, urban adventurers who drifted out at night looking for action with a black man's code to fit their facts. The hipster had absorbed the existentialist synapses of the Negro, and for practical purposes could be considered a white Negro."

Mailer also pointed out that this new breed of "psychic outlaw" could be equally a candidate for the most reactionary or the most radical of political movements. And in the film there is a fascinating scene where Mailer takes on the trades unions on one of the avenues in New York. He tells them that in the past they were a heroic movement - but that now they have become a repressive, stultifying force in society - in particular in the way they are refusing to allow blacks and hispanics to move up society. It is an odd moment because as you watch you realise that it was elements of this rebellious individualism that both Thatcher and Reagan would later harness. And that possibly, if the left had got hold of it earlier, then the history of the West might have been very different.

In the 1970s the phenomenon that Mailer had identified grew massively. And as it did the new cultural bohemians co-opted another outsider culture to give themselves further identity - the gay culture that had risen up in response to homosexual discrimination. Then in the 1980s that bohemian individualism became the driving force that permitted consumer capitalism to reinvent itself - because it offered the ever-multiplying hipsters the objects through which to express their rebellious difference.

Today it is possible to argue that we have all become gay white negroes. We all listen to "edgy urban" music, spend our time in the gym, go shopping and groom ourselves, take lots of drugs, have sex and then spend the rest of the time talking to our friends about the impossibility of finding real love and connection in the world.

Far from an expression of rebellion it has become the conformity of our age. And if Mailer was right - that it was a sensibility originally born out of the existential fears of nuclear holocaust - then it lost its real radical purpose when the cold war ended.

Instead the white negro hipster has actually become one of the central conservative pillars of our time - because their real function is now simply to prop up an increasingly shaky system of credit and rolling consumption.

And one wonders where are the real outsiders of out time? Who are the new "white negroes" of our age?

A year ago I published the story of Murdoch's rise to power in Britain over the past 40 years - as told through the BBC's coverage of him. I thought it would be good to republish it today.

I notice that Lord Justice Leveson brought Woodrow Wyatt's diaries with him into the enquiry today. As you will see from the end of this story he might learn something very interesting about Rupert Murdoch from one of Wyatt's last entries in those journals.

RUPERT MURDOCH - A PORTRAIT OF SATAN

Rupert Murdoch doesn't like the BBC.

And sometimes the BBC doesn't seem to like Rupert Murdoch either.

Following the principle that you should know your enemy, the BBC has assiduously recorded the relentless rise of Rupert Murdoch and his assault on the old "decadent" elites of Britain.

And I thought it would be interesting to put up some of the high points.

It is also a good way to examine how far his populist rhetoric is genuine, and how far its is a smokescreen to disguise the interests of another elite.

As a balanced member of the BBC - I leave it to you to decide.

Murdoch first appears in the BBC archive in a short fragment without commentary shot in 1968. It shows him ambling into the City of London on his way to see Sir Humphrey Mynors who was head of the City Takeover Panel

Murdoch was going to ask Sir Humphrey for permission to take over the News of the World. Then he is interviewed afterwards.

The News of the World was a salacious rag, but it was run by Sir William Carr who was a member of an old establishment family. He had already received a hostile bid from the publisher Robert Maxwell. Carr hated Maxwell because he was not British (he was Czech).

Then Murdoch arrived. He wasn't British either, but he told Sir William he would buy the paper but they would run it jointly together.

Maxwell warned Sir William not to trust Murdoch. He told him - "You will be out before your feet touch the ground".

Sir William replied - "Bob, Rupert is a gentleman"

But Lady Carr began to worry. She took Rupert Murdoch out to lunch in Mayfair. She reported that he had little small talk, no sense of humour and that he had lit up a cigar before the first course.

The BBC got interested in Murdoch - and they put out a profile of him. It was shot with him at work and at home in Australia. It has a great interview with Murdoch's secretary about what a sensitive man he is - and how upset he gets when he has to fire someone.

The News of the World battle ended at a showdown at the shareholders meeting in January 1969. The BBC had a camera inside. Here are some of the shots - again without any commentary.

The shareholders were being asked to accept Murdoch's offer.

It has great bits with Robert Maxwell huffing and puffing about how Murdoch hasn't played by the rules. Murdoch's response - "Yesterday Mr Maxwell called me a moth-eaten kangaroo. I'd like to point out that I haven't yet got to that stage"

Robert Maxwell would go on to become one of the greatest criminals in British business history. And then he would fall off a boat in the Atlantic and drown in 1991

But Robert Maxwell was right in his warning. Within three months Murdoch forced Sir William Carr out - and took over complete control.

Carr died in 1977. Murdoch offered to pay for a memorial service. But a proud Lady Carr refused.

The British establishment decided Murdoch was not a gentleman. And then he did something much worse. He announced he was going to publish the memoirs of Christine Keeler in the News of the World. Keeler was a "model" whose liaison with a government minister John Profumo in 1963 had ruined Harold MacMillan's government.

But since then Profumo had redeemed himself in the eyes of the establishment by going off to work for a charity in the east end of London. So when the News of the World published the sordid details of the affair, the whole of London society was scandalised. Murdoch was unearthing a scandal that should have been dead and buried, and destroying one of their own.

And, they said, he was doing it with the sole interest of lining his own pocket. Murdoch was seen as sleazy and destructive.

And this is where his monstrous image began. The man who had first taught Murdoch journalism on the Daily Express in the 1950s summed it up:

"The trouble is - Rupert was regarded as the Supreme Satan"

And he had also just bought the Sun.

So the BBC decided to make a longer, more probing profile. And to do it they sent a key member of the broadcasting elite - David Dimbleby.

The film is surprisingly fair - given the outrage. Dimbleby puts the accusations to Murdoch, but he also flirts with him, and with Murdoch's wife Anna. It is fascinating to watch Murdoch's face as Dimbleby does this. You can see him beginning to realise just how the British establishment really operates.

The Canadian in spectacles who appears first is Lord Thomson of Fleet - head of a global newspaper empire. He owned the Times in Britain.

The man chairing the editorial conference with Murdoch is the News of the World editor Stafford Somerfield. He was a legendary Fleet Street figure. A few months later Murdoch would sack him.

Somerfield then went off and edited a magazine about pedigree dogs - and became a judge at Crufts.

This rejection by the British establishment was one of the main reasons why Murdoch decided to leave Britain in 1973. He took his family and went to live in New York while still running the News of the World and the Sun in Britain. He talked about his reasons in an interview he gave to a left-wing journalist called Alexander Cockburn in 1976 in the Village Voice.

Although, as the consummate newsman, Murdoch turns it round and portrays it as him rejecting them. And you can see his guiding myth beginning to take shape here - the revolutionary outsider against the decadent British system.

Then in 1981 Rupert Murdoch returned to Britain and took his revenge. He bought the Times.

It was an act that united both the liberal elites and many old Tories in shock and outrage. This got worse when Mrs Thatcher's government allowed the takeover to proceed without it being referred to the Monopolies Commission. Under law this should have happened, but the government excused it with the flimsy excuse that neither the Times nor the Sunday Times actually made money.

There was a growing sense that Murdoch was now manipulating British politicians for his own personal gain. So the BBC decided to investigate Murdoch's business and personal background.

A Panorama was made called "Who's Afraid of Rupert Murdoch?" It was in two parts. First is a film which tells the story of Murdoch's rise to power in Australia, Britain and America. And then he is interviewed live in the studio by yet again - David Dimbleby.

The film is tough. And Murdoch is made to sit and watch it in the studio as he waits for the interview. It lays out and reports all the accusations that would become the foundation for future criticism of the way Murdoch both built and ran his media empire.

-That he takes over intelligent newspapers and turns them into trash. As the ex-editor of the New York Post says - "he took it towards a readership we believed didn't exist"

-That his critics say he turns the news reporting in his newspapers into a propaganda wing of his chosen editorial line, and then uses that to destroy politicians he doesn't like and help elect those he does.

- It describes the scandal in America when Murdoch got a massive favourable loan from the US government just after he had endorsed Jimmy Carter in the New York primary. Murdoch denies there was any connection.

- And it reports the outrage in New York over the sensational way his newspapers reported the serial Killer Son of Sam. Headlines personally overseen by Murdoch that seemed, it was alleged by other journalists, to turn a brutal killer into a celebrity.

- And it gave the American liberals a chance to reveal that they too now hated Rupert Murdoch as much as the British elites. "He is a force for evil" says the head of the Columbia Journalism review rather smugly.

And then Murdoch is given a chance to respond. Here are the parts of the interview where Murdoch takes on those allegations and responds with what was now his central argument.

That he is engaged in a war on elitism - both on journalism in America and the "typical piece of slanting and elitism" that he has just had to watch. Made by the BBC.

It now became his mantra. Anything that was "elitist" could be a legitimate target.

In 1986 Murdoch moved all his operations out of Fleet street to Wapping. The print unions went on strike - only to discover they had fallen into his trap. Murdoch promptly sacked them. The unions, he said, were another part of the decadent elites that were preventing Murdoch performing his proper role - making sure the market system served the people properly.

There was massive TV coverage of the outrage. But the BBC made an interesting programme that looked beyond Murdoch's rhetoric and linked the move to Wapping to what Murdoch was doing in America.

Murdoch had bought Twentieth Century Fox and then, in the months before Wapping, a chain of TV stations called Metromedia (they would become Fox TV). He was massively in debt, and the only way for his empire to survive, it was alleged, was to get more money out of his original purchases - the News of the World and the Sun.

The BBC programme was made by Robert Harris. He went and interviewed one of the American bankers involved in the deal - from Drexel Burnham Lambert - who says that the move to Wapping immediately increased the value of the British papers by over 300%.

Or as one of the union men says in the programme - "British workers are being forced to lose their jobs to fund his investments in America"

In 1989 - on the 20th anniversary of buying the Sun - Murdoch helped write an editorial that trumpeted his vision of himself as a revolutionary:

'The Establishment does not like the Sun. Never has

There is a growing band of people in positions of influence and privilege who want OUR newspaper to suit THEIR private convenience. They wish to conceal from readers' eyes anything that they find annoying or embarrassing.

LIVING LIES AND HYPOCRISY ON HIGH CAN HAVE NO PLACE IN OUR SOCIETY

IT IS THE STRUGGLE OF ALL THOSE CONCERNED FOR FREEDOM IN BRITAIN.'

But the liberal elite were already fighting a counterattack. It had begun with the chat-show host Russell Harty the year before as he lay dying in a hospital bed from hepatitis.

Harty was a homosexual who had been hounded by the News of the World. With his illness this had turned into a media frenzy - with reporters from all the tabloids pursuing him in hospital, posing as junior doctors demanding see Harty's medical notes, and photographers renting a flat opposite his hotel room.

At Harty's funeral in 1988 the playwright Alan Bennett publicly accused the tabloid press of accelerating his friend's death. "The gutter press finished him"

The Sun chose to reply:

'Stress did not kill Russell Harty. The truth is that he died from a sexually transmitted disease.

The press didn't give it to him. He caught it from his own choice. And by paying young rent boys he broke the law.

Some - like ageing bachelor Mr Bennett - can see no harm in that. He has no family.

But what if it had been YOUR son Harty had bedded?'

The BBC decided to quiz Rupert Murdoch. And they chose not David Dimbleby but their main attack dog.

Terry Wogan.

Murdoch was agreeing to interviews at the time because he was promoting his new Sky TV.

It is a very odd episode. Wogan starts off in an embarrassed way - asking Murdoch "is it difficult for you to keep a grasp of reality?". Then he attacks him in a chat show way about his behaviour towards "other chat show hosts" and he manages to get the audience to boo Murdoch.

The only other guest on the programme was someone from the very heart of the British establishment. The Duke of Westminster. Wogan interviews him in a creepy way about the Duke's good works for charity.

A balanced programme.

Here are some parts of the Murdoch interview.

And then came the Sun's distorted reporting of the Hillsborough tragedy which disgusted even some of Murdoch's most fervent supporters .

All this was a disaster for Murdoch the revolutionary. When the 1992 general election began Labour announced that if they won they would introduce new cross-media ownership rules - and force Mr Murdoch to break up his empire.

This would mean he would either have to give up his new dream - the satellite TV station Sky - or he would have to sell his newspapers.

One of Murdoch's biographers says that no other company in Britain stood to lose so much from a Labour victory in 1992 as News Corporation.

And the Sun launched a massive campaign against Labour. Ending on the day before polling with a famous cover. While inside on page three there was an overweight old woman in a swimsuit with the caption - "Here's what Page Three will look like under Labour"

When Murdoch heard the news that John Major had been re-elected he was on the lot at Twentieth Century Fox. He said two words:

"We won"

The key to Murdoch is how you interpret the word WE. Did he mean "We the people" - and that he truly is a populist revolutionary?

Or did he mean by "we" the new financial elite that had risen up in the 1980s that was using debt and junk bonds to break into the old corporations and businesses?

One man who thought he had the answer was one of Murdoch's closest allies who a few years later would come to believe he had been ruthlessly betrayed by Murdoch.

He was the journalist Woodrow Wyatt. Wyatt had been very close to Mrs Thatcher throughout the 1980s and he had become what he proudly called "Rupert's Fixer". But secretly Wyatt was writing a diary every day recording not just his life within the establishment but also his day to day dealings with Rupert Murdoch.

The diaries are wonderful. And in them Murdoch is a dark, silent figure - always listening on the other end of a phone somewhere in America or Australia as Wyatt tells him the inner secrets of the powerful people who run Britain.

But then - in 1995 - Murdoch begins to change. He decides he likes Tony Blair and tells Wyatt he may support him at the coming election. Wyatt can't believe it. He had thought that Murdoch would always support the Conservatives.

And then Murdoch does something worse. He tells the editor of the News of the World to cut back on the column that he had allowed Wyatt to write every week.

Wyatt is in despair. There is a wonderful moment in the diaries when Wyatt sleeps all night on the floor of his study next to the phone waiting for Murdoch to ring.

He never does.

And then - towards the end - Wyatt pours out the truth (as he sees it) about Murdoch. It is in a diatribe to one of Murdoch's American advisers, the economist Irwin Stelzer.

Wyatt cannot believe the treachery. He was the man who fixed it so Mrs Thatcher wouldn't refer the Times purchase to the Monopolies Commission. And now Murdoch is betraying him and turning to Blair.

Like a flash of lightning on a dark night Wyatt believes he sees Murdoch's true relationship to power.

And then in 1997 when Murdoch comes out for Blair, Wyatt has only one line.

But curiously they do.

Both the terrible structural problems and distortions that underly Iraqi society today, and the strange, contorted poses that millions of people perform every day in things like Bikram's Hot Yoga, actually come from the fevered imagination of the British ruling class one hundred years ago.

As they felt Britain's power declining they wanted desperately to go back into the past and create a purer and more innocent world, uncorrupted by the messiness of the modern industrial world - a new Eden forged both by strengthening and purifying the human body and by inventing new model countries round the world.

And we are still suffering from the consequences of that terrible nostalgia.

At the end of the nineteenth century a fanatical craze for physical fitness swept through Britain. Millions of men and women took up gymnastics, body building and other physical exercises.

Such a thing had never happened before - and it was given a name - Physical Culture.

The craze had an almost religious intensity because those who promoted it said that it was the only way to prevent the British nation - and its Empire - from collapsing. Behind this was a powerful belief that the modern world of the 1890s - the teeming cities with their slums and giant factories - was leading to a "physical degeneracy" in millions of people.

It was a fear that had started with the elite who ran Britain's public schools. Matthew Arnold warned of "the strange disease of modern life" with its "sick hurry" and "divided aims". Out of that came a movement called "Muscular Christianity" which wanted to recreate the kind of heroic human being that existed before industry and the modern world came along and corroded everything.

It was a vision of a restored physical and moral perfection in the young men who were going to run the empire. And it involved doing lots of exercises in new things called Gymnasiums. Then liberal reformers got worried about the working classes - convinced that the slums were leading to a "physical degeneracy" . So they persuaded lots more people to do exercises.

Then a figure rose up who united all of this dramatically into a mass movement. He was called Eugen Sandow.

Sandow came from Prussia, he started as a circus and music-hall performer. But then in the late 1890s he invented something he called "body-building". It caused a sensation throughout Europe and America - and he became a massive celebrity because he was seen as the leader of a crusade of Physical Culture that was going to stop the degeneracy that was plaguing Britain.

Here is some film shot by Thomas Edison - showing Sandow in action.

Sandow said that building the perfect body was a way of reconnecting with a pre-industrial time of virile physical perfection. He was very good at PR - and he told a story of how he had gone with his father to see the Greek and Roman statues in Italy. He asked his father why there were no more such men?

His father replied that in those days the rule of the survival of the strongest had not yet been corroded by the dangerous, cushioning effects of "civilisation". There and then, Sandow said, he resolved to lift from himself - and the world - "the stigma of weakness".

And to do that you had to "build" your body to look like this

Sandow also started a magazine called Sandow's Magazine of Physical Culture - to promote what he called The Gospel of Strength. It became the centre of a worldwide movement that incorporated bodybuilding with all sorts of physical exercises.

It was the start of the modern idea of fitness - and at its heart was an almost spiritual vision of restoring a lost wholeness to both human beings and to the world. The American promoter of Physical Culture, Bernarr Macfaddden wrote in 1904

"Our ancestors were strong, virile and conquering because they lived close to Nature and so absorbed her inexhaustible vitality. But we are losing our inherited vitality, slowly perhaps, but none the less surely."

In 1905 Sandow set up "The Empire and Muscle Competition", and then went off on a tour of the world. When he arrived in British India he became a sensation - thousands came to see him in his giant tent.

He had arrived in India at a time of rising tension. There were growing protests against Britain's rule, and Sandow's gospel of strength now began to get mixed up with another ideology - Indian nationalism. In the next twenty years, as Britain's hold over India weakened, the culture of physical fitness that Sandow had brought to the country would re-emerge in a strange mutated form as a way of fighting against British rule.

And in a further mutation this would lead to what we now know as modern Power Yoga.

After the First World War the territories of the old Ottoman Empire were divided up amongst the European powers, and Britain got three provinces in Arabia that would become the new country of Iraq.

Britain had created new countries within its Empire before and it had always started by surveying in extraordinary detail the societies they were ruling - compiling censuses and records of property boundaries and a mass of other details. Out of all that they then built a new administrative system.

It wasn't often very fair or democratic - but it bore some relationship to the reality of existing power structures.

But by the 1920s Britain was bankrupt after the war and couldn't afford such elaborate preparations. Instead a small group of elite administrators were allowed to create a new society out of their imaginations.

And their imaginations were influenced by exactly the same yearning for a return to a pre-industrial rural idyll that had created the Physical Culture movement in Britain.

What the British administrators did was take a romantic vision of a long lost Britain run by feudal landlords and project it onto Iraqi society - where the tribal Sheikhs were seen as being like the British landed aristocracy.

A historian called Toby Dodge has written an absolutely brilliant book called Inventing Iraq. It lays out in clear and very persuasive detail how this group of British Civil servants in Iraq built something that looked like a modern nation - but was in fact a facade. Behind it was really a weird nostalgic myth about Britain.

At the heart of this group was the legendary Gertrude Bell. She wrote the key "Review of the Civil Administration in Mesopotamia" in 1920, and Dodge shows how she, like many of the men working with her, completely distrusted the new modern middle class that had grown up in the cities like Baghdad.

This class had helped run the Ottoman Empire and the British believed that they were tainted - that they had been corrupted by the despotic Ottomans, and that if they were given power they could rise up and become despots themselves.

To prevent this, Bell and the other colonial administrators turned instead to the tribes in the countryside and the Sheikhs that controlled the tribes. The Sheikhs would be a far better alternative - powerful "people of influence" who could help the British run Iraq. They were "true" Iraqis, unscathed by Ottoman influence.

Here is a picture of Gertrude Bell.

What made the rural tribes and their leaders so attractive to the British was the fact that they seemed - in their imaginations - to be just like the stable feudal world of Britain with its rural nobility. The British were explicit about this, the Administrative Report for the Basra Division in 1918 said:

"These landlords are men of gentility and pride, occupying a position of influence and status reminiscent of that of the feudal landlords of English history"

Gertrude Bell was full of the romance of the Sheikhs, she said they were "aristocrats" who managed to keep the collectives they headed in a "natural equilibrium".

Some British administrators in Iraq thought this was mad - that you couldn't transmit authority and order through the tribal system, especially because the sheikhs' political and social power had declined long before the British turned up. It was also sidelining the one group who could help create a proper modern society - the middle class in Baghdad.

But, as Toby Dodge shows, the romantic vision of the sheikh as the linchpin of rural society won out. His judgement is blunt:

"This vision had little to do with the historical or social truth of the society. It sprang in large part from the colonial officials own understandings of the evolution of British society.

To the British the noble bedouin, untouched by all that was negative about the modern day, stood in stark contrast to those who peopled the cities - to those who had succumbed to the temptations of modernity."

If Dodge is right - and his evidence is very powerful - what the British did was create Iraq as an expression of their own fears about what was happening to their own country. They took their worries about the rise of the urban mass, and the horrors of industrialisation in Europe and projected this onto the complex societies that were all mixed together in the nascent Iraq.

They then ruthlessly ignored this complexity and gave a lot of power to the noble, virile sheikhs - who were very like the noble heroes that Eugen Sandow wanted to recreate with his bodybuilding.

Here is part of a film that gives a perfect and vivid illustration of this British romantic view of the Arab tribe as pure, uncorrupted society. It is made by the explorer Wilfred Thesiger who spent the 1940s living among the Marsh Arabs in Southern Iraq, and then with the Bedouin nomads who live in what is called The Empty Quarter that straddles Saudi Arabia and Iraq.

But problems emerged right away.

The British supported the sheikhs who were prepared to co-operate, but there were others that the British deemed "unruly" - and because of this division there was growing anger and resentment among some of the tribes. And from 1920 onwards there were rebellions. But there was also a growing economic crisis in Britain - the defence budget had been cut in half and there wasn't enough money to find the troops that were needed to put down the revolts.

So the British invented what they called "Air Control". It was the first use of aerial bombing to put down a civilian uprising - and it was promoted as both a humanitarian and a moral way of keeping control. The bombers would be clean and precise, hitting only the buildings and fuel stores of the unruly tribes.

The first large-scale bombing was in November 1923 in the Samawah district on the Euphrates. It was against defiant tribes from the Bani Huchaim confederation. A British Special Services officer called John Glubb had done a reconnaissance and worked out who he thought were the sheikhs who led the tribes. His operations map showed:

"the location of the villages belonging to the Shaikhs and Headmen whose influence among the tribes rendered them particularly suitable for attack."

In doing this Glubb was following the simplified British vision of Iraqi rural society. In fact the society in Samawah turned out to be more complicated than he imagined. When the identified sheikhs were told to surrender or face bombing two of them came to the British and told them that they didn't have the power to make anyone surrender.

But the British thought they were being evasive - and the bombing went ahead. It was the shock and awe of its time. The RAF planes came in and bombed the villages, the people fled and returned as darkness fell. Then that night the planes came back with incendiary bombs and caught the villagers. An RAF report said that it was:

"to do away with the idea that they (the targets) will ever have any period of peace once an attack has begun."

The RAF's conservative assessment after the attack said that approximately 100 civilians had been killed and six villages destroyed. There was a lot of public concern in Britain about this new tactic, and in the face of this, John Glubb later claimed that only one Iraqi had died.

Glubb was one of the central military figures in Iraq - and his actions showed just how dangerous the simplified British vision of Iraqi society could be.

Here is a photograph of him:

Much later - in 1981- Glubb appeared on a very odd BBC chat show. It was called Friday Night Saturday Morning, and the theme was "The Arab People". It had a very strange collage of guests - first a Saudi prince comes on to defend his regime, then the romantic novelist Barbara Cartland dressed all in pink sits next to him and explains how every woman wants to have sex with an Arab sheikh.

Then John Glubb joins them to describe enthusiastically his bombing campaigns in Iraq in the 1920s. He starts by talking about the origin of "Air Control" but then slips away into a practiced, humorous after-dinner set of anecdotes about how the tribes were like little children who spent their time raiding each other - and had to be bombed to make sure they "played fair" like in cricket.

In an extreme, surreal way the programme illustrates the weird myth of "the noble sheikh" that the British had projected onto Iraq - and the extreme violence needed to sustain that myth.

By the late 1920s there was a craze for Physical Fitness sweeping through India. It was something completely new to Indian society and it was led by a famous body-builder and gymnast called Professor K.V. Iyer. He had been inspired by the western ideas of Eugen Sandow and, like Sandow, he had turned exercise into both a physical and a moral duty.

Iyer modestly described himself as having "a body which Gods covet" and gave himself the title "India's most perfectly developed man"

What made Iyer's vision of a strong body so attractive to many Indians was not just physical. It was also a way of expressing the growing nationalism and hatred of British rule. Indian nationalists were very aware of the way their colonial masters dismissed all Indians as a weak and degenerate people - Baden Powell famously called them "enfeebled". A powerful body was a way of challenging that in dramatic physical terms.

In 1927 the popular journal Vayayam - The Body Builder - said its mission was "to uplift India from the mire of physical decadence."

K.V. Iyer was very aware of the paradox - that Indians were using European ideas of physical exercise to challenge their European colonial masters. And at the end of the 1920s he took his theories of body-building that were based on Western models and fused them with the spiritual ideas of Yoga. The aim was to create what one of Iyer's closest collaborators called "A Physical Culture Religion" which deliberately had roots in India's ancient past. They called it "The Yogic School of Physical Culture".

It was something very new - that had very little to do with traditional Yoga as it had been practiced for centuries. Yet it is the root of almost all the modern Yoga practiced today in Europe and America.

Such an idea is heresy to what are called the "Yoga Fundamentalists" in the west today who portray Yoga as having a special antiquity that goes back thousands of years. But recently a Yoga teacher and academic called Mark Singleton has written a fascinating and gripping book that challenges that idea head on. It is called Yoga Body.

Singleton goes back to the India of the 1920s and 1930s and shows in forensic detail how modern Yoga was constructed out of Western ideas of gymnastics and a modern Indian political nationalism. He points out that traditional Yoga has very few poses - and most of those are variations on the seated meditation posture. For hundreds of years, Singleton says, yoga was not about physical fitness but a system of meditation and philosophical enquiry.

Here is some footage - from BBC news in 1957 - of this new kind of physical yoga being displayed to the new leader of an independent India - Pandjit Nehru.

And in a brilliant piece of detective work Singleton goes on to show how in the 1930s a completely fictitious spiritual history was created for this new kind of Yoga - which then allowed it to be sold back to the west as something ancient and mystical.

It happened in the Jaganmohan Palace in Mysore. The Maharaja was a fitness fanatic, he installed a gymnasium and invited K.V. Iyer to come and teach his body building there. As Singleton shows, in the next door room was an unknown yoga teacher called T. Krishnamacharya who then proceeded to take the yogic physical culture that Iyer had invented - and push it much further.

Here is a picture of the palace.

Out of it came a radically physicalised form of yoga which is the basis for almost all the modern forms of yoga like Power Yoga that have grabbed the western imagination.

What made this so attractive to the west was that Krishnamacharya said that his system was five thousand years old and based on an ancient text called the Yoga Kurunta. He had first heard of the text, he said, when he was taught the system by a guru high up in the mountains in Tibet. He had then returned and "discovered" a copy of the five-thousand year old Yoga Kurunta in a Calcutta library, which he then transcribed.

Strangely no-one has ever seen the original text. Unfortunately when his followers asked to see it Krishnamacharya told them that it had been eaten by ants.

Singleton makes it clear that the real inspiration was far more likely to have been the body-building contortions and gymnastic exercises going on next door in the gym of the Mysore Palace.

Here is some footage of one of Krishnamacharya's followers - who was also his brother-in-law - called B.K.S. Iyenegar who was the person who brought this yoga system to Europe and America and made it famous. The first is an early exhibition he did for the BBC in 1966. It is followed by a wonderful scene of the same Mr Iyenegar on a BBC evening magazine programme from 1981 getting the presenter to do this "ancient spiritual exercise". Iyenegar was by then 63 years old. Obviously this kind of yoga works.

Gertrude Bell died in Iraq in 1926 - having taken an overdose of sleeping pills. Noone knows whether it was suicide or not. But what is known is that she had come to realise that the British attempt to build a nation out of Iraq had failed. In a letter she wrote:

"There's no getting out of the conclusion that we have made an immense failure here. The system must have been far more at fault than anything that I or anyone else suspected. It will have to be fundamentally changed and what that may mean exactly I don't know."

In many ways the story of Gertrude Bell and her family is also the story of the fall of the British Empire. Her grandfather had been a wealthy industrialist who had made his fortune in iron and steel. He then became a powerful Liberal party politician helping to create the global vision of Empire under Disraeli.

Gertrude was one of the generation who then struggled in the 1920s to keep that global vision alive in the face of economic crisis and political and public opposition in Britain - and failed.

Strangely it was Gertrude Bell's half-sister, Mary, who would show the way forward to the next stage of this global vision - a mystical vision of the world in which individuals around the globe were no longer dominated by political power - but instead united by a vague, spiritual force. It was the New Age philosophy - and Yoga was going to play a central role in this new ideology.

Mary Bell's eldest child was Sir George Trevelyan who would become one of the founders of the New Age movement in Britain. The central guiding idea of the movement was that the world was moving towards a new age in which the fragmented and divided societies and nations would die away. It would be a "oneness" - a restored unity with all the people of the earth, with nature and within your own body.

The earliest and most powerful concrete expression of these ideas was the Findhorn Foundation. It was a rural community in Scotland whose aim was to try and create a model for this new kind of unified world. Sir George Trevelyan helped create the Findhorn Foundation - and tirelessly promoted it as a vision of an alternative future for the world.

I want to show a programme that the BBC made in 1973 about Findhorn. It was taped in their community hall where the founders and many of the members of Findhorn were asked to explain their vision, questioned by a very sympathetic presenter called Magnus Magnusson.

It is incredibly funny and wonderfully bonkers, but it is also very touching. I particularly like the middle-aged, very respectable man who says that he often meets "the Great God Pan" on the streets of Edinburgh - and then says that the God Pan is sitting in the audience tonight - "somewhere towards the back".

And the man in charge of the Findhorn garden is just brilliant - both in his fashion choice and his conviction that the vegetables he grows know telepathically what he is thinking and can feel his love for them.

"The vegetables are happy to be eaten because it is an expression of love. It is a wholeness, a oneness. I am at one with the lettuce I eat, especially after I've eaten it."

Here he is - full of vegetables.

Sir George Trevelyan is sitting in the front row next to the two founders of Findhorn - Eileen and Peter Caddy.

What is fascinating is that none of these people are hippies - they are the disillusioned children of the British empire. The Caddys had met in the early 1950s when both were stationed on an RAF base in Iraq - it was RAF Habbaniyah, the airfield from where many of the Air Control raids had taken off. Both were disenchanted with their lives and had come back to Scotland to try and build an alternative kind of world.

Here is a frame grab of Peter and Eileen.

Sir George, the Caddys, and the others sit there describing eloquently and sincerely how they want to telepathically get in touch with nature to create a new Eden and build "a great harmonious oneness that links us all".

Everything they say is suffused with a yearning desire to recapture something that has been lost.

It's as if what they are really doing is creating a fantasy global empire that is run by what Peter Caddy calls "different administrative levels of natural spirits". An empire that is populated by thinking, telepathic, vegetables that are happy to be eaten - just like the happy natives that were content to be ruled by the white men that loved them for their simplicity and innocence.

Behind the New Age movement in Britain was the same belief that had driven the physical culture movement sixty years before - that the modern world and above all industrialisation was corroding both the moral and physical fitness of human beings. The aim was to restore a new unity of mind and body.

But the movement couldn't turn to the old ideas of health and fitness because in the 1930s they had become inextricably linked with nationalism - above all in the Nazi cult of physical fitness and the superman.

And that is where Yoga came in because it offered a system of physical exercise that also promised to create a spiritual oneness with the mind. It was physical exercise cleansed of all political connotations - and based instead on a powerful mystical tradition that went back five thousand years (even if the ants had eaten all the evidence for that).

And Yoga really took off in the New Age movement. It was one of the physical activities at Findhorn - and by the 1970s it had swept through the West. Here is a wonderful bit of film. In 1978 the BBC sent Sir George Trevelyan to report of the Festival For Mind and Body at Olympia in London. And one of the first thing Sir George wants to show you is Yoga.

I wish more reporters were like Sir George - I love his style, especially the way he quotes Wordsworth's pantheistic vision of the world in the middle of Olympia.

In Iraq Britain's failed attempt to create a modern state in the 1920s has haunted the country ever since.

In 1958 there was a military coup which began a period of bloody violence that led to the country being taken over by the Baath party in 1968. When they took power the Baathists deliberately set out to try and dismantle "premodern" tribalism. They did this both to try and finally modernise and strengthen the country - but also because the tribalism was so linked in their minds to collaboration with British imperialism.

The Baathists tried out experiments with the collectivisation of land ownership in 1970 - and then started to nationalise land in 1971.

But in the late 1970s the structure of power began to strangely mutate - and as Toby Dodge argues in his Inventing Iraq - it moved backwards towards a copy of the very same tribal structure of patronage that the British had instituted.

This happened because of the rise to power of Saddam Hussein and the Tikritis within the Baath ruling elite. As power became increasingly personalised around the figure of Saddam, the power of the Baath party came to depend on the al-Bu Nasir tribe - and within that the Beijat clan group.

And in the process Saddam began to do exactly what the British had done in the 1920s. He turned away from the urban political elite (in his case bloodily - by ruthlessly executing scores of senior Baath party members that he thought were threats to him) and moved towards using the tribal system. He set out to co-opt other tribes - and to try and break the power of others

Then - after the Gulf War in 1991 - Saddam went further. He effectively recreated tribal networks and tribal "recognized sheikhs" all across Iraq who were given resources and power in return for loyalty to him. Just like the British.

And as Dodge points out - that structure continued after 2003

"It is these very same "recognized sheikhs" that the British and American forces have begun to look to for the cost-effective provision of order in the post-Saddam era.

If one were able to pick up Iraq like a good piece of china and turn it over, it would bear the legend: 'Made in Whitehall, 1920'."

Here is footage of Saddam from 1981 that shows how he was reinventing this structure. There is footage of him going to rural villages, sitting next to the headman, and promising them stuff in return for their loyalty.

It is followed by a extraordinary description of how he invented himself almost as a super sheikh - his personal telephone number listed in the Baghdad phone book - that anyone could call and talk to - just like going to see the headman.

Meanwhile Yoga conquered the western world.

But the history of Yoga is just as convoluted and contorted as the positions its followers adopt. For much of the ideas behind it were initially born as attempts to morally reinvigorate the minds and bodies of those who ran British Empire. Those ideas then swept through India and became part of a nationalism that challenged Britain's rule. They then were sold back again to the west in a new form - linked to a mysticism that gave a purpose and meaning to a nostalgic post-imperial generation.

Today yoga has morphed once again. Much of the new age mysticism linked to it has fallen away, and in an age of intense individualism where people increasingly feel disempowered, the human body has become the last territory individuals feel they have control over. It is the Empire of One - and Yoga is the administrative system that controls it.

Here is Jerry Hall reporting on the latest fashionable version - Bikram's Hot Yoga - and meeting Bikram himself who claims to have "800 plus" schools across the world. She also then visits an Ashram in India and is puzzled to find that their Yoga isn't really like what goes on in the Hollywood Hills.

But in the 1970s everything changed. For that was the moment when religion was deliberately brought into politics in both countries with the aim of using it as a revolutionary force. And those who did this - Khomeini in Iran, and right-wing activists in America - were inspired by the revolutionary theories and organisations of the left and their ambition to transform society in a radical way.

I want to tell the forgotten story of how this happened - and how in the 1980s both the Americans and the Iranian idealists came together in a very odd way - with disastrous consequences.

In the early 1970s in Washington a small group of young conservative activists came together to try and change American politics. They called themselves the New Right and they were convinced that unless they did something drastic, the liberals and the left-wingers in America were going to take over the country.

One of the leaders of the New Right was a man called Paul Weyrich, and in the wake of the student revolts of 1968 he infiltrated the meetings of left-wing grassroots organisations. He was astonished by the amount of planning and tactics that he saw and he realised that the conservative movement in America was completely unaware of all this. The right, he said, were still trapped by the belief that people would simply vote for them because they were right.

So the New Right set out to organise a new grassroots movement that could counter the left's success. They had all sorts of discussions and during one of them Weyrich pointed out that there were millions of Americans who were socially and culturally very conservative but who never voted. They were the religious fundamentalists and the evangelicals - a vast segment of the population who believed that they should never get involved in politics.

Weyrich realised that if you could activate the fundamentalists and the evangelicals then the New Right could create an incredibly powerful force. But the problem was how to persuade them. The fundamentalists were driven by pietism - the belief that a true Christian should not only devote their life to god, but also turn their back on the secular political world. They should live the good life through their own actions - and forget about politics.

Ironically it was the liberal left that offered Weyrich the way to activate the fundamentalists. Since the late 1960s the left had pushed through reforms on all kinds of moral issues - gay rights, abortion, sexual discrimination. This had shocked the Christian heartland of America because it was politics attacking and undermining the very beliefs through which they lived their private lives.

The final straw came when President Carter abolished the charity status for the fundamentalist religious schools. This really hurt because they thought Carter, an evangelical, was one of them. But Carter was of the old school - he believed that religion should be separate from politics.

So in May 1979 Paul Weyrich and four other young activists drove to the Holiday Inn in Lynchburg Virginia to meet one of the most powerful evangelical pastors in America, Jerry Falwell. Like a number of other pastors, Falwell had his own television network and millions of followers. What happened at that meeting would shatter the pietism of millions of fundamentalist Christians and bring them - and their beliefs - into the heart of American politics.

I interviewed Paul Weyrich and another of the New Right group, Morton Blackwell, about that meeting. Here they are - describing what happened. It begins with Weyrich telling how he infiltrated the left. Weyrich was a fascinating man (he died in 2008) - a conservative revolutionary.

At the same time - in early 1979 - the Ayatollah Khomeini led a revolution that toppled the Shah in Iran.

Khomeini did this by completely transforming Shia Islam. It was a religion that for hundreds of years had taught its millions of followers to turn their backs on politics and power. Khomeini had turned this upside down - and had brought Shiism into the heart of politics.

Back in 1963 Khomeini was just another conservative cleric living in the City of Qom, but then the Shah launched the White Revolution which was supposed to modernise Iran. Khomeini was horrified because the programme was going to emancipate women, swear in elected officials on any holy book - not necessarily the Koran, and worst of all it threatened to take away the clergy's very large landholdings.

Here's an image of the future of the Shah's revolution. Girls running nuclear power.

But the problem was how to challenge the Shah? Shiite Islam had a quietist attitude towards politics. One of its main ceremonies is "Shiite lamentation", where the faithful ritually flagellate themselves. Throughout Shiite history the clergy have made this the symbol of a retreat from the world - and above all from politics and power. The people must wait in a world full of shadows and evil - for the return of the twelfth imam. This meant that political power was evil and debased, and you must have nothing to do with it.

Khomeini decided to overturn this - and to do it, like Paul Weyrich in America, he turned to the ideas of the political left.

In the 1960s an Iranian sociologist called Ali Shariati had become fascinated by the writings of the Third World revolutionary, Franz Fanon. And when Shariati translated Fanon's writing into Persian he used the language of Islam - so marxist terms like "the oppressors" became "the arrogant" while "the oppressed" became "the weak" or "the disinherited".

For Khomeini this was the key - and in 1970 he gave a series of lectures that took Shariati's attempt to fuse revolutionary Marxism and Islam and used them to portray a new vision of Shia Islam. Your duty, Khomeini said, was no longer to remain passive but to seize power and drive out the wicked and corrupt ruler. It was an extraordinary move, because Khomeini was exploding one of the fundamental ideas of his religion.

You don't just sit around waiting for the Messiah. You fight - and you take power now. Led by the clergy.

Khomeini lifted a lot from Shariati, but it was also driven by his powerful personality and his brilliant use of the media. Here is is film of Khomeini in exile in Paris in 1977 as his ideas were taking hold in Iran. It is followed by some film rushes of the extraordinary mass demonstration that happened in Tehran on the 29th March 1978 - the slogans on the banners show the fusion of left wing revolutionary ideas and Islam. As the Shah says in the film, Khomeini had combined "the red and the black".

A few weeks later violent revolution began - and I have included in the rushes some great and very brave reporting in the midst of the fighting by the BBC reporter Richard Lindley.

In America the politicisation of religion had taken off in a big way. Throughout 1979 The Revd Falwell travelled the country contacting, he claimed, 72,000 pastors. He showed them how to mobilise their millions of followers and how to register them to vote. Falwell also worked with the New Right to use Direct Mail to dramatise the moral issues - and to provoke.

When gays were allowed to lay a wreath on the tomb of the Unknown Soldier in Washington, Falwell sent out a warning to the Moral Majority followers:

"That's right - the gays were allowed to turn the tomb of the Unknown Soldier into:
THE TOMB OF THE UNKNOWN SODOMITE!"

But the question was - who should the newly radicalised fundamentalists support in the 1980 presidential election? The Religious Right prepared a Presidential Biblical Scorecard which was sent out to millions.

It scored all the candidates on the great moral issues - abortion, homosexuality, national defence, and many others. Jimmy Carter didn't do very well.

But the question was really decided at a dramatic mass meeting in Dallas. It was called The National Affairs Briefing and was sponsored by the Religious Roundtable - a coalition of religious groups. All the candidates for President and other political figures were invited to come and explain their views on religion - but only one turned up, Ronald Reagan.

The meeting's other aim was to show just how many leading pastors now believed that evangelical religion should become involved in politics. Along with Falwell, leading televangelists like James Robison and Jimmy Swaggart whipped up the 17,000 strong crowd in the Dallas Arena - in front of 50 million television viewers.

Other leading pastors, like Billy Graham, refused to come. They hated what was happening. One of them, a Baptist called James Dunn gave a brilliant quote:

"We've got a bunch of TV preachers who want to establish a theocracy in America, and each one of them wants to be Theo."

And then Reagan made his speech.

Here is James Robison at the meeting followed by Reagan, a moment that many in the movement say was the turning point. I have also included Jimmy Swaggart attacking those who say religion should not be involved in politics - because it is really funny and shows just how powerful and confident this movement was back then.

But as all this was happening in America, the battle that was taking place in the Iranian revolution over religion and politics spilled over into American politics - and things began to get very complicated.

And its victim was the hapless President Jimmy Carter.

Unlike previous US presidents, Carter didn't like the Shah of Iran. The CIA had told him horror stories of what the Shah's secret police were doing to Iranian dissidents. And Amnesty were publiciising the same thing - like the use of bacon slicers to cut off prisoners' hands bit by bit. Carter didn't like this, he believed that America should promote human rights around the world and he publicly criticised the Shah.

But not very strongly. Carter said that criticism of the Shah's secret police was "perhaps sometimes justified", while he continued to give Iran vast amounts of weapons.

In 1977 the BBC were making a sycophantic documentary about the "life of Washington's first lady" - Rosalynn Carter. They were filming in the White House when the Shah of Iran came to visit. Carter had promised he was going to tell the Shah he should try and liberalise his country. Unfortunately thousands of Iranian exiles didn't think this was enough and they turned up outside the White House to protest, and Rosalynn's plans started to go wrong

When the Iranian revolution happened, President Carter tried to contact what he believed were "the moderates" in the revolution. The embassy in Tehran opened a dialogue with the liberals who had allied themselves with Khomeini - and who now wanted to transform Iran into a democracy. The most important was the new Prime Minister Mehdi Bazargan.

But Khomeini wanted to get rid of these liberals because they were opposed to his idea of the new political structure of the Islamic Republic of Iran - where absolute power would be given to "the Guide", which meant Khomeini himself. The liberals saw this as the restoration of a dictatorship.

So Khomeini and his supporters manufactured a crisis. On November 4th 1979 500 "students in the line of the Imam" (ie followers of Khomeini) stormed the American embassy and took the diplomats hostage. There are stories that a young Ahmadinejad was one of the students, but no one has proved this and he denies it.

There is a great book written by Massoumeh Ebtekar who was one of the invaders of the embassy. She describes how when they began to explore, the students found tons of shredded documents lying discarded on the floor and in the barrels of the shredders. One of the invaders, an engineering student called Javad thought that the shreds from each document must have fallen together - and so it might be possible to rebuild the documents.

"He was a study in concentration - bearded, thin, nervous and intense. These qualities combined with his strong command of English, his mathematical mind and his enthusiasm, made him a natural for the job.

One afternoon he took a handful of shreds from the barrel, laid them on a sheet of white paper and began grouping them on the basis of their qualities.

"After five hours we had only been able to reconstruct 20-30% of two documents. The next day I visited the document centre with a group of sisters. 'Come and see. With God's help, with faith and a bit of effort we can accomplish the impossibe', Javad said with a smile."

A team of twenty students then went to work to reconstruct all the papers - in the end they published 85 volumes of them. The documents revealed the deep and cynical involvement of America in supporting the Shah throughout the 1970s. They were the Wikileaks of their time, for they showed how the CIA had worked closely with SAVAK - the hated and vicious Iranian secret service.

The students renamed the embassy "the nest of spies" - and quite a lot of the hatred and distrust of America that has pervaded Iran ever since comes from those reconstructed documents.

And what's more the documents also helped Khomeini destroy his liberal allies, because they revealed that, since the beginning of the revolution, President Carter had been talking to "the moderates". Khomeini seized on this and used it to force out and arrest all those in the new government who wanted a democracy. They were traitors because they had been corrupted by the Great Satan.

Khomeini then used the embassy crisis - fuelled by the hatred of America - to build his vision of a radical theocracy in Iran. It had an enormous effect on the Presidential campaign in America because it made Carter look impotent, especially when his mission to rescue the hostages failed dramatically with helicopters crashing and burning in the Iranian desert.

To give a sense of the drama and uncertainty of that time - here are some sections from the BBC News Comp tapes of the time - 1979-80. They were the 2-inch video tapes onto which raw material coming over satellite from Washington and Tehran was first dumped. I have kept them as they are - and they give a very good sense of the complexity and dislocation of what was happening. They include footage the Iranian students shot as they invaded, and then some video of what they found inside the embassy - the secret communications equipment, and then they find the shredders.

In November 1980 Ronald Reagan won the presidential election. Millions of newly radicalised Christians voted for him. Reagan would have won without their votes, but the New Right had awoken a powerful force that now came into Washington - an evangelical conservatism that wanted to change the world, not just keep it the way it was, as traditional conservatives always had.

By 1980 the idea that left-wing politics could change the world was finished and over - throughout the western world. And in a strange way these new conservative radicals were the last spasm of twentieth century revolution - created out of left-wing tactics borrowed by the New Right - and then fused with fundamentalist anger.

But the problem was that almost immediately Reagan ignored them. Although in speeches he paid lip service to their fury over subjects like abortion - he did almost nothing to remake America into the morally good country they sought. And the religious right and their supporters were frustrated and angry.

But they still had hope in foreign policy. Like the fundamentalists, Reagan saw foreign policy not as realpolitik, but as a global battle of good against evil - and he backed the idealists in his administration who wanted to support what they called Freedom Fighters in countries like Nicaragua.

But this would lead the American religious idealists into a very weird situation - they would become the allies of the religious revolutionaries in Iran.

Because Khomeini's revolution was also having problems. The country was facing an economic disaster and the millions of poor people who had created the revolution were finding that their prospects hadn't really changed. While the intellectual leftists who had supported Khomeini had turned against his idea that he should be in charge.

So Khomeini simply annihilated the left. He killed them - or forced them to confess to treachery on TV from prison.

Then Iraq invaded Iran - and hundreds of thousands of the most devoted and active revolutionary militants were sent off to become cannon fodder. They were giving their lives to defend the revolution - but their deaths also removed the growing threat from this group as Khomeini's revolution failed to solve the economic and social problems.

And by 1984 Iran had become a very dark and strange place. Any idea of using religious energy to change the world was gone - and faced with the appalling butchery in the war, Iranian Shiism found it's way back to the old idea of martyrdom, but in a horrific way.

The historian of modern islamism, Gilles Kepel, described what happened to that young revolutionary generation.

"The appalling butchery of the war against Iraq gave the younger generation of poor Iranians an incentive to return to the former tradition of martyrdom.

No longer at issue was the transformation of the world, for the revolution had clearly failed to satisfy that expectation. Rather the young men developed a new desire - a longing for death - as a response to the failure of Iran's revolutionary utopia and the pressures of the war with Iraq.

The Shiite death wish took on massive dimensions with the sacrifice of the bassidjis at the front. The colunteers wrote letter and last testaments to their families, asserting their longing for death. What these tragic documents describe in religious terms is no less than the political suicide of the young urban poor of Iran in the 1980s."

In 1984 the BBC made a two-part documentary recording this dark, strange Iran. It is a brilliant film - it shows just what Kepel describes, hundreds of young men being bussed off to the front every day, welcoming the fact that they are all going to die.

The giant fountain in the mass war cemetery spouts blood-red water. While in cool, white offices, very young children are taught to embrace the idea of martyrdom by a spooky cleric - and are given toy models of the US space shuttle to reward them. And the revolutionary guards spend their time driving around policing their neighbours' morals, and hunting down ill-veiled women (bad hejabi).

Here are some sections from the film. I really like the way it is made - refusing to bow to the normal hysterical news style. Its calmness evokes the growing darkness brilliantly.

And into this weird, dark world came an equally weird American - called Colonel OIiver North. He was a radical Christian fundamentalist who wanted to save his, and Reagan's, global revolution through an audacious and, in retrospect, completely crazy plan.

North was high up in the National Security Council and had been running a secret programme to help the Contra guerrillas in Nicaragua - it was part of what he saw as an epic battle of good vs evil all around the world. But Congress had found out about it - and stopped him.

So in 1985 North, began to build an amazing scheme. He knew that the Iranians were desperate for weapons in their war against Saddam Hussein, so he proposed to sell them thousands of missiles, then take the Iranian money and use it to secretly fund the Contras. The Iranians would also persuade the Hezbollah guerrillas in Lebanon to release American hostages.

North, along with the National Security Adviser, Bud McFarlane, told Reagan that this would also be a way of opening a dialogue with "the moderates" in the Iranian regime. And out of this North built an epic vision whereby this would allow America to defeat the extremists in Iran, end the Iran-Iraq war, and root out all Islamist terrorist networks in Europe and around the world.

A journalist called Ann Wroe wrote a fantastic book in the 1980s about the Iran-Contra affair. What she describes is an incredible comedy - somebody should make a drama about it.

North gave everyone and everything code names:

Missiles were "dogs"
The airport was "a swimming pool"
Iran was "apple" - so Tehran airport was "apple swimming pool"
But confusingly in another code sheet Iran was "tango"
Israel was "banana"
The United States was "orange"
Hostages were "zebras"
So a typical message in North's notebook was:
"IF THESE CONDITIONS ARE ACCEPTABLE TO THE BANANA, THEN ORANGES ARE READY TO PROCEED"

But it got more confusing because North kept on giving himself different code names. Initially he called himself "Wagner", but then he began sending messages about the plan signed "Steelhammer". Then he called himself "Colonel Goode", while his right hand man, General Secord was codenamed "General Kopp". And then he started calling himself "Mr Green".

Here are two orange zebras:

At times North got confused about who he was. When he boarded planes he couldn't remember what name he was on the passenger list - and had to go through all of them until he got it right.

North and McFarlane started meeting with representatives of the Iranian regime in great secret in places like Frankfurt. They were convinced they were dealing with "moderates", but no one could define what a moderate was in Iran - especially when it began to seem that "conservatives" in the theocratic regime were also "radicals".

The Iranians got thousands of missiles - and three hostages were released. But then Hezbollah kidnapped three more hostages - and the Americans were back at zero again.

But North was convinced that it would work because it was the meeting of two groups - from America and Iran - who both devoutly believed that their political aims had a grander, religious purpose. He flew to Tehran to try and solve it. North sat listening to his Iranian contact talking emotionally about Martyrdom. North replied

"Because I am a Christian, I understand and believe that when one dies in faith he will spend eternity in a far better place"

The Iranians got lots more missiles, North got more money for the Contras - but no more hostages released. He got desperate and arranged another meeting with the Iranians in Frankfurt. North took a bible with him in which he had persuaded President to write an inscription - and he gave it to the Iranians saying:

''We inside our Government had an enormous debate, a very angry debate inside our Government over whether or not my President should authorize me to say 'We accept the Islamic Revolution of Iran as a fact. He (the President) went off and prayed about what the answer should be and he came back with that passage I gave you that he wrote in front of the Bible I give you.

And he said to me, 'This is a promise that God gave to Abraham. Who am I to say that we should not do this?' ''

At one of their meetings, an Iranian came up to North's right-hand man, General Secord, and said:

"What's with this guy North? We just left a country full of mullahs, and what do I find here but another goddam mullah."

Then - at the end of 1986 - North's mad scheme was exposed. There was an enormous political scandal that nearly brought Reagan down. And the revolutionary visions of the religious right were finished.

The Iranians made great play of how mad Oliver North was. The then speaker of the Iranian parliament, Rafsanjani, held up North's bible for the world to see:

But the religious right in America didn't go away, instead - just like in Khomeini's Iran - it has mutated since the late 1980s into a rigid moral police force that has become an iron cage that possesses American politics and stops it progressing.

And that is what someone like Rick Santorum is. He's no revolutionary. He's just a conservative. But whatever happens to his campaign, the religious right is an active force in American politics. In particular they are among the keenest to bomb Iran.

Meanwhile in Iran there are parliamentary elections - but it's a contest between different factions of conservative religious fundamentalists, with the opposition excluded.

Forty years ago, in both America and Iran, religion was brought into politics as a revolutionary force - fuelled by a vision that it could be used to transform the world. But now, in both countries, that power has mutated into a backward-looking and hysterical conservatism that is doing its best to remove both countries from the dynamic force of history.

And look what happened in 1987 to one of the great leaders of that revolution in America - Jimmy Swaggart. But even when he faced his downfall, he was a great performer.

One September night in 1945 three British mathematicians and astronomers went to see a new film at a cinema in Cambridge. It was called Dead of Night. It was a series of ghost stories told by a group of people gathered together in a farmhouse. The stories are linked by a device of a central character who is convinced that he has experienced the whole situation in the farmhouse before. In the end he murders another of the group - but then wakes up from this terrible dream.

That morning the telephone rings, he is invited down to the farmhouse, and the whole story, or dream, starts all over again.

The scientists loved the film, and they sat discussing its circular structure. One of them suggested that it could be the model for how the whole universe really worked. That, although the universe was expanding, it was also constantly renewing itself - to maintain itself in a steady state.

Out if this came what was called the "Steady State" theory of the universe. It was going to dominate scientific thinking for the next twenty years, and it would also make one of the three scientists very famous.

He was a very difficult and argumentative man called Fred Hoyle - and the story of what happened to him and his idea is odd and funny - and also shows how science can often add a spurious certainty to the stories that modern societies tell themselves.

I also want to tell the story of two of the men behind the film Dead of Night - because both of them were convinced that the certainties of the post-war years had trapped Britain in a narrow bubble that was preventing it from seeing the world as it really was.

And we may still be in that bubble.

Fred Hoyle was one of the first scientists to become famous on television and radio. It was because he told a dramatic story about the universe - about how amazing it is, and the extraordinary discoveries that astronomers like him were making.

Ever since the 1920s scientists had realised that the universe wasn't static - it was expanding. There was a furious dispute about what this meant. One group of cosmologists said it meant that the universe had begun with an enormous explosion. Hoyle thought this was ridiculous, and he derisively gave his opponent's theory a name. He called it "the big bang" on a radio programme in 1949.

Hoyle thought this idea was silly because it meant that everything that now exists in the universe would have had to have been created in that one explosive moment. Hoyle believed that the universe had no beginning and no end - and that fiery stars throughout the cosmos were continually creating new matter that filled up the universe as it expanded.

And in 1948 Hoyle published a paper that was more than just a piece of scientific theory. It amounted to a new philosophical description of the universe, and it captured the public imagination. Two years later the BBC invited Hoyle to give a series of lectures on the radio, and millions listened to his dramatic vision.

Underpinning it was Hoyle's belief that mathematics has an objective truth to it - but that truth is something that we as humans can only dimly perceive. What astronomers were starting to find, Hoyle believed, was just a tiny part of something truly awesome. A giant mathematical plan to the universe that we will only ever understand a tiny part of.

Here are bits from a couple of films the BBC made about Hoyle and his ideas. He is obviously a very difficult character, but he has a great way of expressing himself. I love his description of what human beings are like when faced with the mathematical plan of the universe. They are, he says, like "fish a mile or two off Yarmouth". They can glimpse Yarmouth, but they will never come near comprehending it properly.

The most dramatic part of Hoyle's theory was the way it challenged our concept of time - that all things must have a beginning and an end. Hoyle dismissed his opponents' belief in the big bang as being a simple reflection of the deep human desire to see everything in the world as stories.

In another BBC programme he put it bluntly:

"The reason why scientists like the "big bang" is because they are overshadowed by the Book of Genesis. It is deep within the psyche of most scientists to believe in the first page of Genesis"

And the reason that the film Dead of Night had such an effect on Hoyle was because it too has no beginning and no end.

Behind that structure were two fascinating men in the British film industry of the 1940s and 50s who consciously wanted to challenge the happy stories that post-war Britain was telling itself.

They were Robert Hamer and Alberto Cavalcanti. Not only in Dead of Night, but in other films they made, Hamer and Cavalcanti both set out to puncture what they saw as a naive, simplistic vision of human beings, and of society, that had emerged in post-war Britain.

Both Hamer and Cavalcanti were intellectuals of a generation that had been profoundly shocked by the second world war. Not the second world war that we remember today as a simple story of the triumph of good over evil. But something profoundly chaotic, a moment in history when all the comforting stories fell away and millions of people faced a dark and frightening chaos. What that generation learnt was that when that happens anything is possible. It is terrifying because it leads to unimaginable horrors, but it is also exciting because there are no boundaries and you can do whatever you want.

Much of that complex view of human beings was forgotten or hidden away at the end of the war. But I think it came back in some of the films - those made in America in film noir, and in Britain above all with the films of Robert Hamer.

Robert Hamer was a sardonic, disillusioned man who worked as a director at Ealing Studios. In contrast to the happy, naive output of Ealing - like Passport to Pimlico - Hamer was blunt about his ambition. He told the wife of Ealing's head of publicity:

"I want to make films about people in dark rooms doing beastly things to each other"

Hamer directed the section of Dead of Night where a man is given an antique mirror by his fiancee. When he looks into it he sees another, older, room that begins to possess him. It turns him into a violent, depraved man, and he tries to kill his fiancee. It is the story of the overwhelming power of madness and destructive passion. And order is only restored at the last moment.

But Hamer's masterpiece was a film he made two years later in 1947 - called It Always Rains on Sunday. It is a wonderful, powerful film. The central character is Rose, she had been a barmaid in a pub in Bethnal Green, but now she is married to an older man. Then suddenly her old lover Tommy turns up. He is on the run from prison and he pleads with Rose to shelter him.

Rose loves Tommy and she deceives her family - hiding Tommy in the upstairs bedroom.

The film's power comes from the intense mood it creates. Rose, played by Googie Withers, beautifully expresses the feeling of numbed desire that then breaks out in the dark, claustrophobic rooms of the east end house when she smuggles Tommy in. Passion that smashes through all the naive fantasies of post-war Britain.

But Tommy is not good - he loves Rose, but he is also brutally interested in his freedom, and Rose finds herself betrayed. All the stories have been torn away - both happy family life, and passionate love. She tries to commit suicide, is saved and returns to her family that has now become a prison.

It is impossible to give a proper sense of the film from a clip - you really have to watch the whole thing.

Michael Balcon, who ran Ealing studios, didn't like all this complex pessimism, and he stopped many of Hamer's future projects. I really like the one which was going to be the story of a young man who gets falsely accused of murder, is tried and is acquitted. But in the process he has become intoxicated with being in the news headlines, so he decides to commit a murder in order to experience it all over again.

Hamer made one other masterpiece - Kind Hearts and Coronets. On the surface it was a jolly comedy, but underneath Hamer wrote it as a vicious black satire about how a horrible lower middle class couple lie, cheat and murder their way to the top of British society. They have no goodness as characters - yet Hamer makes you like them, and root for them.

And then Robert Hamer became a self-destructive alcoholic.

I have stumbled upon a wonderful programme the BBC made in 1977 which tells the story of what happened to Hamer. It is a monologue done straight to camera by Pamela Wilcox. She was the daughter of another British film studio boss, Herbert Wilcox. In the late 1950s she met Hamer and fell in love with him, and they started living together.

By now Hamer drank all day. In 1960 he was given one last chance, to direct a film called School for Scoundrels, but he collapsed on the set. He and Pamela Wilcox then started living an extraordinary life, both of them were drinking heavily and bit by bit they fell into their own private hell.

Wilcox tried to escape - only to find that she fell further into what she describes in the programme as "limbo-land". Like the heroine of It Only Rains on Sunday, she found herself without any comforting stories, facing only the terrifying chaos that is existence.

It is a really moving story - and she tells it very vivid way. You can see how annoying and self-destructive she must have been. But you also really like her and get caught up in her story.

I have cut the beginning where she tells of her time living in Hollywood in the early 50s. It picks up when she discovers that her American husband has a lover - and she returns to England.

The vivid story that Fred Hoyle told about the universe not only caught the public imagination but it also promoted the idea that mathematics was somehow the key to understanding everything.

Hoyle had a mantra that he repeated in all of the programmes he appeared on:

"If there is a god, then Mathematics is God. The basic laws of physics, insofar as we have any concept of god, is God."

What Hoyle meant by this is that the science of mathematics was not just a construct projected onto the stuff of the universe - but that the pattern of order that physicists and astronomers like him were uncovering was mathematical.

It was a very powerful idea - and many other disciplines turned to mathematics in the post-war era to try and give themselves a power and dignity that they felt had been missing from their "science".

One of these was economics - and out of the economists' attempts to "mathematicize" their discipline would come another great story of our time. The conviction that the economy could be organised in such a way as to achieve its own "steady state".

In the 1930s most economists didn't bother with mathematics - except for a few numbers in tables. But after the war an economist called Paul Samuelson decided to take the mathematical methods developed to study the laws of thermodynamics and apply them to economics. It was an extraordinary move - because what Samuelson said was, that just as a thermodynamic system seeks to achieve equilibrium, so too does an economy.

Getty/Time&Life/YaleJoel

Samuelson called it the "Ergodic Hypothesis", and the image he gave was of the economy as being like a giant pendulum - that wherever you started its swing from, the pendulum would always want to settle back on the same fixed point. To put it in his language, the ergodic hypothesis was:

"a belief in a unique, long-run equilibrium independent of the initial conditions"

The simple phrase "initial conditions" is the key - because it means that whatever you do to the system, or whatever turbulence hits the system, it will always want to return to the same stable position. In other word - history doesn't matter. Again there is no beginning and no end. There is just the equilibrium - just like the steady state of the universe. And the economists' job was to help the economy achieve that equilibrium.

It had an enormous effect on political ideas about how to manage economies in the 1950s. The theories were further simplified so it became a technocratic dream that promised the politicians an economy that would expand but would also remain stable.

A few dissenting voices in the 50s said this was a pseudo-science. That what people like Samuelson were really doing was telling stories about the world in mathematical form. But the substitution of numbers for words seemed to make the stories more valid.

Here is an extract from a film from the Pandora's Box series that I made about the history of those economic experiments in 1950s and 60s Britain. It begins with an economist called Bill Phillips who built a giant computing machine run by water to demonstrate how to make an economy expand, yet not run out of control.

In 1961 the Steady State theory began to fall part. The attack was led by Fred Hoyle's sworn enemy - a Radio Astronomer called Martin Ryle. Where Hoyle was stubborn, Ryle was angry, sometimes so angry that he was made to work in a different building in Cambridge from his colleagues to avoid violent arguments breaking out.

In 1961 Ryle gave a press conference proudly announcing that he had discovered pulsating things called quasars far out in the universe. This proved that the steady state theory was wrong, he said, because if it was correct then qasars should be everywhere - not far out in the distance receding from us.

And then, four years later, came the killer blow. A couple of scientists in New Jersey called Penzias and Wilson had started picking up a hissing noise on a giant radio antenna that they were using. At first they thought it was the pigeons inside the antenna, so they went out and shot all the pigeons.

But the hiss continued, and they finally decided that they were picking up something much bigger. It was called Cosmic Microwave Background Radiation, and it was a faint glow of radio light that seemed to fill the universe.

This kind of radiation had been predicted by the big bang scientists because it was the radiation left over after the early stages of the universe's formation. Now it had been found, and their theory conquered the scientific world.

This is their big radio antenna, after the pigeons had gone.

But Hoyle didn't believe it. He accepted that the radiation raised serious questions, but he wasn't convinced that the big bang was the answer. He thought that the scientific establishment were like fish two miles off Yarmouth believing that they now understood what Yarmouth was really like.

To express his anger Hoyle wrote a TV drama series for the BBC called A for Andromeda. His aim was to show how scientists, and the politicians they entrance, can become possessed and corrupted by what seems to be a pure mathematical theory.

A for Andromeda tells the story of how astronomers pick up radio signals coming from outer space. They decode the signals and from them learn how to build a giant computer. The computer then electrocutes one of the scientists - Christine, played by Julie Christie. But it then recreates a perfect clone of her.

The hero - John Fleming - becomes convinced that the computer is being used to take over the earth by the aliens who sent the signals. But none of the other scientists see this. Then the government becomes obsessed with the computer because it seems to answer all their needs. It starts to run their defence systems - and then it offers to run the country's economy, and make it a rational system.

So Hoyle's hero - who was a model of Hoyle himself - has to save the world.

All the episodes have been lost except for one, the fifth out of six. It is very much in the model of John Wyndham science fiction, but Hoyle's fury burns through, especially in the passionate speech of his hero as he sets out to destroy the computer. It's great - and it was very popular.

The most frightening section of Dead of Night is the story of the ventriloquist who becomes possessed by his dummy. Michael Redgrave plays the ventriloquist called Maxwell Frere who has a dummy called Hugo - and the film follows Frere as he descends into madness. Hugo becomes a vicious, horrible and cruel figure that seems to control his master. At one point Frere says quietly to a rival ventriloquist: "You don't know what Hugo is capable of".

The director of this section was Alberto Cavalcanti. He was originally from Brazil where his father was a mathematician. But in the 1920s Cavalcanti had gone to Paris where he became deeply involved with avant-garde filmmaking. Then in the 1930s he came to England and worked in the early documentary film movement.

Like Robert Hamer, Cavalcanti thought that the British had a dangerously false vision of themselves - a twee artifice of forced jollity. He expressed this most powerfully in 1942 in a film he made for Ealing Studios called Went the Day Well? It tells the story of how a group of Nazis, disguised as British soldiers take over a beautiful English village.

The films starts as a piece of war propaganda. The nazis are vicious and sadistic. Then the villagers start to fight back, but instead of being noble and kind they become even more violent - and what's more they start to really enjoy it.

Here is a brief clip, but it will give you a sense of what Cavalcanti was up to.

Cavalcanti later said of the people in the film: "People of the kindest character, such as the people in that small English village, as soon as war touches them, become absolute monsters"

His point wasn't the simple oh-dearist lament that "people are bad", it was that humans are very complex, and that they have all sorts of dimensions and capacities that the simple stories leave out. For Cavalcanti, and many of his generation who experienced the second world war, post war Britain was possessed by a false and shallow cheerfulness.

In the story in Dead of Night the ventriloquist is possessed by the opposite. His dummy is a dark, bitter, destructive character that for Cavalcanti represented the dimension of human beings that was being brushed under the carpet and suppressed.

What Cavalcanti was saying was that just because we fought a good war, it doesn't necessarily mean we are good people.

In the 1990s I made a series called the Living Dead. The first film was about how that complex and extraordinary experience that many people lived through in the second world war was wiped away and forgotten immediately after the war. And how it was replaced by a simple story of the Good War.

The film argued that we are still possessed by that simplistic myth of goodies and baddies - a myth that subsequently has been the main driving force behind humanitarian interventions from Kosovo to Iraq and Libya. That myth says that if we liberate the people from their evil oppressors, then they will automatically become like us - good people.

Here is a part of the film. It begins with the story of the film that the American prosecutors showed at the Nuremberg trials in 1946 . It was called the Nazi Plan, and helped create the idea of the Good War. Set against this is the personal experience of some of the Americans who had fought in the war. One of them is a very interesting man called Paul Fussell who went on to become a well-known writer and critic in post-war America

And economics was rapidly becoming another central part of that simplified post-war world.

In the early 1970s there was a terrible battle between the man who had bought mathematics into economics - Paul Samuelson - and the free-marketeer, Milton Friedman.

Friedman said that Samuelson's mathematical models - that showed governments what to do to make their economies stable - were completely useless. Samuelson had been promoting something called the Phillips Curve which had been created by the same Bill Phillips who had built the giant economic water machine.

The Phillips Curve showed, Samuelson said, that if governments let unemployment rise then inflation would inevitably fall. Unfortunately it didn't. In the 1970s both unemployment and inflation went rocketing up, and none of Samuelson's followers could explain why.

Milton Friedman said that the solution was simple. Governments should stop trying to manage their economies and other than controlling the money supply they should just let things rip. This became the cornerstone of Mrs Thatcher's policies in the 1980s, which culminated with the deregulation of the financial markets in the City of London in 1986 - which was called the Big Bang.

Unlike the cosmologists and their Big Bang, the free market economists still believed that this explosion would lead to a stable equilibrium.

They had a simplified, mathematical vision of human beings as creatures who logically analysed everything in the market, and then reacted as if they were computing machines. Few people at the time said that this might be an area that maths didn't really have anything to do with, and that the scope of equations like the Black-Scholes model - which made derivatives trading predictable - might be very limited.

Here is a montage of the TV news reports leading up to the Big Bang in the City in 1986. It's a great picture of the old world that was about to be swept away - and the new one that was coming. Plus a wonderful early example of fear journalism - promoted in a very funny interview by a detective from the Fraud Squad.

But in the 1970s the mathematics behind the cosmologists' theory of the Big Bang started to lead to some very weird places. The man who caused the problems was Stephen Hawking.

Hawking had begun his work as a cosmologist by studying the big bang. He looked at black holes - they are what happens when a star collapses in on itself. Thus they are a sort of reverse of what is supposed to have happened at the origin of the big bang.

In the 1970s Hawking showed mathematically that black holes were eating information. Put simply this meant that stuff in the universe was disappearing into the black holes. Under the theories of quantum physics this was impossible, and lots of other cosmologists got very angry. But Hawking's mathematics were so good that no one could disprove it.

It was called The Black Hole Information Paradox.

Hawking's argument had massive implications. As he himself pointed out - it meant that there was no certainty any longer in the universe, it undermined the whole idea of cause and effect, and you couldn't predict the future with any certainty, or even be sure about what had happened in the past.

It was pretty bad.

So lots of other cosmologists set out to destroy Hawking's proof. They were led by a man called Leonard Susskind who publicly declared war of Hawking. It led to a wonderful, vicious scientific battle - and a few years ago Horizon made a film about the thirty year War of the Information Paradox.

The proof that Susskind finally comes up with is extremely odd. It involves accepting that if you as an individual fell into a black hole - it would look to those watching you from outside as if you had been torn to pieces. But in reality you would still feel as though you were you - even though you had been stretched around the edge of the black hole to become a two-dimensional version of yourself. Just like a cinema film.

You begin to wonder whether the mathematics isn't leading the cosmologists to tell stories that are far odder than any science fiction. I particularly love the scientist who explains that there are trillions of black holes throughout the universe. He says there might even be black holes inside his own head.

Here is a part of the Horizon programme.

Meanwhile things hadn't been going very well for Fred Hoyle. He had been appointed the head of the Institute of Theoretical Astronomy in Cambridge, but he was so difficult and argumentative that he was forced to resign. He then turned his back on the scientific establishment and went to live in the Lake District - where he continued to write science fiction with his son who was a national pistol champion.

But Hoyle still managed to fascinate the BBC - and he persuaded them to make a ninety minute film about his ideas, and dramatise one of his stories. The drama is truly one of the worst things I have ever seen on television - so I thought I would show part of it.

A beam from outer space has made time slip on earth. But in some places it makes time go forward, while in others it goes backwards. Time has effectively disintegrated and two scientists move separately through this dislocated world until they both end up in Mexico - but a Mexico way in the future.

Hoyle also put forward even odder scientific ideas. He said that he had proved that intelligent life on earth had originally come from outer space - in the form of bacteria carried on meteorites. When other scientists attacked this - he became even more convinced.

Hoyle had succumbed to the thing he had attacked other scientists for back in the 1960s - he had become possessed by his own stories. He had become a fish that thinks it knows what Yarmouth is like - and Hoyle's vision of Yarmouth had turned out to be a very strange place indeed.

Coincidentally, at the very same time, the BBC made an episode of the Holiday Programme about Great Yarmouth. It was presented by Joan Bakewell. No fish, however good their mathematics, could have ever imagined this.

Meanwhile Hoyle's enemies - the proponents of the Big Bang theory - had been trying to deal the fundamental problem with their theory.

What came before the Big Bang?

And out of this has come a new idea in recent years which is called The Big Bounce. This says that before the initial explosion there was a contracting universe just like ours which collapsed in a Big Crunch, and then exploded out again. What this implies is a cycle - with the universe exploding out, contracting and then collapsing until it explodes - and starts all over again.

Which is just like the structure of the film Dead of Night.

You have the funny feeling that you have been here before.

And since the economic crash of 2008 Britain seems to be returning to an older form of society and politics.

I'm told on reliable authority that the servants at Chequers feel more at home today than they have for a very long time. The cycle has returned to its conservative origins.

Maybe it was all a dream.

Getty/AFP/CarldeSouza