Danny Hylton's 89th-minute winner sent 10-man West Ham crashing out of the Carling Cup and put League Two side Aldershot into the second round.
The Hammers led through a curling shot from Junior Stanislas but had debutant Callum McNaughton sent off for a professional foul after the break.
That changed the game and Luke Guttridge equalised from close range after Ruud Boffin denied Alex Rodman.
Hylton snatched victory from 12 yards out after a goalmouth scramble.
That gave Aldershot their first win in this competition since the club reformed in 1992 and returned to the Football League in 2008.
But for the third time this season West Ham were left to rue conceding an important late goal at Upton Park.
Hammers boss Sam Allardyce made nine changes from the team that were held at home by Leeds in the Championship on Sunday and initially his side struggled to settle.
It took took a superb save from Boffin to deny Jake Taylor when he ran through but West Ham weathered the storm and took the lead with a strike of real quality from Stanislas.
DEAN HOLDSWORTH FACTS
Shots boss Dean Holdsworth is a lifelong West Ham fan
Holdsworth considers Sam Allardyce his mentor after playing under him at Bolton for four years
The Shots were given hope when McNaughton scythed down Hylton after 47 minutes.
And Dean Holdsworth's side piled on the pressure before Guttridge pounced to level with 12 minutes to go, reacting quickly after Boffin blocked Rodman's shot.
With West Ham hanging on, Michael Rankine and Jermaine McGlashan went close to winning it for Aldershot before Hylton found the bottom corner with a low angled shot.
Matt Taylor came close to equalising in stoppage time but could only fire into the side netting after latching on to a flick.
West Ham United 31 Boffin 02 Reid 03 McCartney 18 Faubert 37 McNaughton Dismissed 04 Nolan 12 Barrera (Taylor - 53' ) 25 Stanislas 11 Carew (Ilunga - 50' ) 19 Sears 30 Piquionne (Nouble - 69' ) Substitutes 42 Cowler 23 Ilunga 14 Taylor 16 Noble 44 Moncur 24 Nouble 46 Hall Aldershot Town 01 Young 02 Herd Booked 03 Straker 05 Jones 11 Rodman 15 Morris 23 Vincenti (McGlashan - 65' ) 04 Guttridge 08 J Collins 20 Taylor (Rankine - 55' ) 10 Hylton (Pulis - 90' ) Substitutes 21 Warner 06 Brown 07 McGlashan 14 Pulis 16 Montgomery 09 Rankine 12 Bubb