A late equaliser from Swansea striker Danny Graham capped a frantic spell of three goals in six minutes and booked the Welsh club an FA Cup replay with Arsenal.
The Gunners' eight-year wait for a trophy seemed set to continue as Michu put Swansea ahead two minutes after coming off the bench.
But Lukas Podolski, another substitute, levelled with nine minutes remaining and left-back Kieran Gibbs crashed in a spectacular volley two minutes later.
Arsenal's joy was short-lived, however, as Graham was left unmarked at the far post to lift the ball into the roof of the net and keep the hosts in with
a chance of a fourth-round trip to Brighton.
- Arsenal have not failed to make the fourth round of the FA Cup since 1996
- In three previous FA Cup meetings between the sides Arsenal have won two and Swansea one.
- Arsenal have won the FA Cup 10 times, Swansea have never reached the final.
There was still time for both sides to create chances to win a match that started slowly but ended in a frantic exchange of goalmouth action.
Sixth in the Premier League, out of the Capital One Cup and drawn against Bayern Munich in the Champions League last 16, Arsenal were desperate to keep their most realistic chance of silverware alive.
But when Michu came off the bench to fire Swansea ahead just before the hour, another disappointment looked on the cards.
As Nathan Dyer's header bounced just inside the Arsenal half, Michu was the quickest to react and lifted the ball over Gunners defender Per Mertesacker.
That gave him a clear run at Wojciech Szczesny and Michu showed characteristic calmness to slot past the goalkeeper with a side-foot finish reminiscent of his two goals in Swansea's 2-0 Premier League win at the Emirates Stadium in December.
Michu's goal - his 15th in 24 appearances this season - came just as Arsenal were enjoying their best moments of the match.
Olivier Giroud twice forced saves from Michel Vorm with driven efforts from inside the area.
In between those shots, Aaron Ramsey screamed for a penalty when he fell as he passed Dwight Tiendalli, although replays suggested the Welshman tripped over his own feet.
After a toothless first-half display that hardly troubled the home side, Arsenal suddenly found some urgency and directness.
And like Michu, substitute Podolski wasted little time in making his presence felt after replacing Ramsey.
The German showed a striker's instinct to swivel and shoot in one smooth motion and find the bottom corner.
Two minutes later, the tie swung in Arsenal's favour as Gibbs exchanged passes with Mikel Arteta and crashed in a volley from a tight angle.
But when Kemy Agustien's corner fell to the feet of Sung-Yeung Ki, the South Korea international showed exemplary composure to set up Graham, who was left in far too much space at the far post.
The striker took a touch and lifted the ball over a diving Szczesny.
There was still drama to come in the closing minutes as Laurent Koscielny's header was palmed away by home goalkeeper Vorm. The rebound fell perfectly for the Arsenal defender but he fired wildly over the bar.
And in stoppage time, Swansea right-back Tiendalli found himself in space inside Arsenal's box but drove a low effort into the side netting.
The replay will take place at Emirates Stadium on Wednesday 16 January.
- 01 Vorm
- 02 Bartley Booked
- 04 Chico
- 21 Tiendalli
- 33 Davies
- 07 Britton (Agustien - 69' )
- 12 Dyer
- 15 Routledge (Pablo - 56' )
- 20 De Guzman (Michu - 56' )
- 24 Ki Sung-Yeung
- 10 Graham
- 25 Tremmel
- 16 Monk
- 22 Rangel
- 09 Michu
- 11 Pablo
- 26 Agustien
- 17 Shechter
- 01 Szczesny
- 03 Sagna
- 04 Mertesacker
- 06 Koscielny
- 28 Gibbs Booked
- 08 Arteta
- 10 Wilshere
- 14 Walcott
- 16 Ramsey Booked (Podolski - 72' )
- 19 Cazorla
- 12 Giroud
- 36 Martinez
- 05 Vermaelen
- 11 Santos
- 07 Rosicky
- 15 Oxlade-Chamberlain
- 22 Coquelin
- 09 Podolski