But was isn't known so widely is that General Lee's family came from Shropshire, and the family home still exists.
For 500 years, the Lee family owned a sizeable chunk of the county in the parish of Alveley, near Bridgnorth.
The family, originally-named de la Lee and probably of Norman descent, lived in Coton Hall from the 1300s onwards. The tombs of of two Lees with effigies are in Acton Burnell Church.
And it's only because the present-day Coton Hall was put up for sale early in 2003 that the Lees of Shropshire came to light again.
Present day Coton Hall was built soon after 1800 for Harry Lancelot Lee, in the Georgian style. At the time the estate ran to 5,000 acres.
Although the present building is only some 200 years old, the Lee family's connections with the land go back 1,000 years. The previous building on the site was also called Coton Hall - and it was from here that Robert E. Lee's ancestors left for America in the 1600s.
|Another view of the building|
They originally went there to trade, and one or two returned to England after a few years, but one branch forged new lives for themselves in the young country, acquiring land and power.
Two of them, Richard Henry and Francis Lightfoot Lee, were the only brothers to sign the Declaration of Independence.
General Robert E. Lee's father was 'Light Horse Harry' Lee, a famous soldier of the American War of Independence, where he was known for his courage in fighting the British.
And by a bizarre paradox, he may well have been reponsible for the deaths of soldiers from Shropshire - elements of the 53rd Regiment, which later became the Shropshire Regiment, were all but wiped out and the remainder captured at the Battle of Saratoga in 1777.
Light Horse Harry resigned from the army as soon as the British were defeated, and settled down to raise the family that included the young general. Well, maybe that's a little kind - before long he deserted Robert and his mother and left them to fend for themselves in Virginia.
|General Robert E. Lee|
Harry bolted to the Caribbean in search of fortune.
Robert E. Lee went into military college at West Point and became a career soldier in the U.S. Army. He graduated from the West Point military school with not a single demerit point - something no-one has ever done before or since.
He joined the U.S. Army but offered his services to the breakaway Confederates when the American Civil War in the 1860s. This made him a traitor in the eyes of the Union, but Lee sided with the Confederate states because his home state of Virginia was one of them.
|Inside Coton Hall|
After four years of war it was Lee who signed the surrender. Lee narrowly avoided trial for treason, but was instead stripped of his rights as a citizen. Despite this he refused to be bitter, doing his utmost to embrace the new United States until his death in 1870.
Today Lee is perceived as an American hero, and not just an icon for Virginia or the southern states whose troops he led. He was finally pardoned of any wrongdoing more than 100 years later by President Jimmy Carter.
According to the previous owners of Coton Hall, several of Lee's descendents have been to visit and cast their eye over the ancestral seat.
|Inside Coton Hall|
But little remains of the house that Robert E. Lee's ancestors would have known.
In the grounds of Coton Hall is one of the last remnants of the early buildings - the ruins of a chapel that probably dates from the 13th Century.
But it's underground where the strongest traces of the old Coton Hall remain. The house's cellar is two storeys deep and in the lower of the two levels includes the entrance to a tunnel.
According to the estate agent FPD Savills, the tunnel runs all the way to Alveley village two miles away, although it's been concreted off beyond the chapel for safety reasons.
|Inside Coton Hall|
Coton Hall passed out of the Lee family when Harry Lancelot Lee died in 1821 and the house was immediately sold, ending the Lees' long association with this part of the world. In 1878 the chapel roof collapsed and all the Lee monuments were moved to Alveley church.
The house itself was extended in about 1860, when a new wing and an Italianate tower were added, but apart from that the house has survived remarkably well - all the fireplaces and cornices are original, for example.
The house, including the 6.5 acres of land it stands in, was sold in 2003 with a guide price of £1.25 million.