Cillian Sheridan and Liam Kelly were on target as Kilmarnock won at Celtic Park for the first time since 1955.
Celtic were a shadow of the side that impressed against Barcelona and fell behind as Sheridan capitalised on a defensive lapse late in the first half.
Emilio Izaguirre felled Rory McKenzie on 62 minutes and Kelly smashed the resultant penalty past Fraser Forster.
While the home side were poor, Kilmarnock were well worth their historic victory.
Neil Lennon's side remain on top of the Scottish Premier League but their lead has been cut to two points, while Kilmarnock have leapfrogged Motherwell and Hearts to move into sixth place.
The visitors were a little quicker out the blocks than their hosts, with James Dayton looking lively, though Forster remained untroubled.
Similarly, Cammy Bell was seeing little action, though he would have been grateful for the intervention of Ryan O'Leary who, after 16 minutes, stuck out a leg to deny Miku the opportunity to head in Izaguirre's cross from close range.
Ten minutes later, Celtic should undoubtedly have gone in front. Kris Commons swung in a cross from deep, Miku headed it invitingly into the path of Joe Ledley, but the Welshman - from just a few yards out - nudged it wide of Bell's left-hand post.
Killie created a presentable chance of their own just after the half-hour mark when Dayton delivered the ball in from the left towards the front post where Sheridan was lurking, but Forster was alive to the danger and knocked it away.
The first time Bell was called upon to make a save was five minutes before the break, as he beat away a Charlie Mulgrew free kick but the opening goal came at the other end.
Adam Matthews was closed down by Dayton and played a wayward pass to Efe Ambrose who, caught unawares, succeeded only in playing the ball off Sheridan.
The former Celtic striker was aided by a bit of backspin on the ball, which convinced Forster he wouldn't get to it first, and Sheridan calmly went round the keeper before virtually walking the ball over the line.
Celtic top-scorer Gary Hooper was introduced for Beram Kayal at the start of the second half and he almost drew his side level within seven minutes.
Ledley fed Forrest on the right and his low ball was clipped just wide by the striker.
Almost immediately, Dayton raced towards the edge of the Celtic box before unleashing a fierce drive, which was brilliantly turned behind by the diving Forster.
Bell dealt with another Mulgrew free kick moments before Kilmarnock increased their lead.
McKenzie showed great skill on the edge of the Celtic penalty area before being clipped from behind by Izaguirre as he shaped to shoot.
Referee Crawford Allan pointed to the spot and Kelly blasted the ball past Forster to put Kilmarnock within touching distance of a momentous victory.
Izaguirre thought he had made amends with a quarter-of-an-hour remaining, but he was ruled to have been offside as he finished off a pass from Pat McCourt.
Then substitute Tony Watt hammered the ball over the crossbar from six yards after Forrest's cutback fell kindly for him.
Watt's next effort was more threatening, but his powerful drive from inside the box was well parried by Bell.
Celtic Park emptied long before full-time as the home fans registered their displeasure. The opposite was true of the small band of visiting fans, who celebrated the end of one of Scottish football's longest-standing hoodoos.