Northern Ireland suffered embarrassment as Luxembourg clinched only their fourth win in a World Cup qualifier.
Martin Paterson fired the visitors ahead but Lars Gerson hit the bar before Aurelien Joachim levelled with a deflected shot before the break.
Stefano Bensi's low shot put Luxembourg ahead, although Gareth McAuley headed in just minutes later to make it 2-2.
Mathias Janisch secured the shock win three minutes from time when he slotted in from eight yards.
Luxembourg - who had last won a home World Cup qualifier in 1972 - were first to threaten at a rain-lashed Stade Josy Barthel when an unmarked Lars Gerson headed over from a corner.
Northern Ireland made a sluggish start but a sharp move put the visitors ahead against the run of play on 14 minutes, with Paterson's first touch from Shane Ferguson's pass giving the striker space to fire his third international goal powerfully into the corner.
Paterson saw a lob drop over before the dangerous Bensi blasted wide from a free-kick.
Gerson almost levelled on 29 minutes when his curling shot struck the crossbar with Carroll beaten, while the visiting keeper was at full stretch to keep out a Bensi drive.
Northern Ireland winger Niall McGinn was forced off with an ankle injury with his replacement Michael O'Connor coming on for his first international appearance in three years.
Oliver Norwood saw his volley saved by Jonathan Joubert before Joachim's fortunate injury-time equaliser, his shot deflecting off Daniel Lafferty and looping into the net after a classic counter-attack.
Luxembourg made a strong start to the second half as Northern Ireland's nerves started to show - and Bensi drilled a 16-yard free-kick just wide after Carroll handled a mis-hit back-pass from McAuley.
Jonny Evans fired wide as the visitors enjoyed their first period of superiority from the hour mark.
Billy McKay wasted an excellent chance after 69 minutes, the substitute put in by Paterson's flick and he rounded Joubert before slicing into the side-netting.
The impressive Jamie Ward sent a shot over before Bensi gave Luxembourg - whose momentum had been waning - a 2-1 lead with a low shot into the bottom corner from outside the area in the 78th minute.
McAuley headed in from a Norwood free-kick four minutes later to make it all-square again, but the hosts were not to be denied a memorable victory.
Northern Ireland had lost all their shape and when they failed to clear a deep corner, Janisch was perfectly placed to slot in the 87th winner, as the good work done by Michael O'Neill's side by beating Russia last month and running Portugal close last week evaporated.