Kilmarnock shocked Rangers and breathed new life into the title race as Manuel Pascali's header gave them a first home win over the Ibrox side since 1994.
Ally McCoist's team were looking to equal an 82-year-old club record by remaining unbeaten on the road for a 15th consecutive game.
However, Kilmarnock had other ideas and Pascali's 80th-minute header from Garry Hay's lofted pass secured victory.
Rangers now lead Celtic by four points after their first loss of the season.
Within the first minute, debutant Sone Aluko supplied Steven Davis inside the box, but his cross from the right for Nikica Jelavic was just too strong and the Croatian could not connect in front of goal.
Aluko looked in the mood and played in Jelavic who hit the post, although the offside flag had gone up for a tight decision.
Kilmarnock, forced to substitute the injured midfielder Danny Buijs for Danny Racchi after seven minutes, briefly threatened from a rare foray upfield as Pascali saw a header blocked from a corner.
Kenny Shiels' side were soon in trouble again as goalkeeper Cammy Bell was challenged by Jelavic as he prepared to kick clear, but the ball bounced out for a throw-in.
The Ayrshire side slowly grew in confidence as the game remained goalless and Rangers were dealt a blow as the injured Kyle Lafferty had to be replaced by youngster Kane Hemmings.
The Rugby Park side's Gary Harkins went down after a shoulder-to-shoulder tussle with Kirk Broadfoot in the penalty box, but the referee was unmoved.
At the other end, McCoist's side had a shot from Hemmings which stung the palms of Bell.
Rangers then had a strong penalty claim on the brink of half-time when Hay took out Aluko in the box in a goalmouth scramble before McCulloch's effort hit the side netting.
Kilmarnock made a strong start to the second period and Dean Shiels' through ball found Harkins only for Steven Whittaker to block his effort and concede a corner.
From the resultant set-piece by Shiels, the ball cracked off Jelavic's head and smacked against his own team's crossbar with Allan McGregor beaten.
It was the Ibrox side's turn to threaten next as Hemmings released Aluko, whose powerful effort was cleared off the line by James Fowler with Bell beaten.
The action switched to the other end after 65 minutes when Liam Kelly's defence-splitting pass sent Paul Heffernan one-on-one with McGregor, but the Rangers No 1 stretched out his left hand to divert the ball clear.
McCoist's men had a fantastic chance to take the lead when handed a free kick 20 yards from goal, but Davis could only fire straight into the opposition wall.
Aluko still had plenty of energy left and his left-foot shot drive trundled into the box and hit the base of the post, but Hemmings could not convert the rebound.
The miss was to prove costly as Kilmarnock swept upfield and Hay lofted a hopeful ball into the box from around 25 yards.
Aluko failed to step out with the rest of Rangers' defence and Pascali connected to head past McGregor and put the home side ahead.
Substitute Alejandro Bedoya shot straight at Bell from inside the box in the closing seconds as Kilmarnock held on for a historic victory.
Live Text Commentary
Full Time The referee ends the match.90:00 +2:55
Allan McGregor takes the direct free kick.90:00 +2:55 Booking
Booking Booking for Dean Shiels.90:00 +2:50
Dean Shiels gives away a free kick for an unfair challenge on Steven Davis.90:00 +2:25
Dean Shiels takes a short corner.90:00 +2:17
Shot by Gary Harkins. Blocked by Kirk Broadfoot.90:00 +0:52
William Gros is caught offside. Allan McGregor takes the free kick.89:10 Substitution
Substitution William Gros comes on in place of Paul Heffernan.88:53
Shot from 12 yards by Alejandro Bedoya. Save made by Cameron Bell.84:17 Substitution
Substitution John Fleck on for Sone Aluko.82:57
Gary Harkins takes a shot. Save made by Allan McGregor.82:11
Mohamadou Sissoko concedes a free kick for a foul on Nikica Jelavic. Dorin Goian restarts play with the free kick.81:09
Dean Shiels restarts play with the free kick.81:09 Booking
Booking The referee books Steven Whittaker for unsporting behaviour.81:07
Foul by Steven Whittaker on Gary Harkins, free kick awarded.79:53
The assist for the goal came from Garry Hay.79:53 Goal scored
Goal! - Manuel Pascali - Kilmarnock 1 - 0 Rangers Manuel Pascali scores a headed goal from close range. Kilmarnock 1-0 Rangers.79:18
Dean Shiels takes a inswinging corner from the left by-line played to the near post, Nikica Jelavic makes a clearance.78:18
The ball is crossed by Sone Aluko, Alejandro Bedoya has an effort at goal from inside the six-yard box that strikes the post.77:30
Gary Harkins concedes a free kick for a foul on Lee McCulloch. Dorin Goian restarts play with the free kick.
Lee McCulloch takes a shot from just outside the penalty area which clears the crossbar.75:07
Direct effort from the free kick comes in from Nikica Jelavic.73:09 Substitution
Substitution David Silva on for Danny Racchi.73:09
Free kick awarded for an unfair challenge on Steven Whittaker by Mohamadou Sissoko.71:10
Steven Whittaker gives away a free kick for an unfair challenge on Rory McKeown. Dean Shiels takes the free kick.70:27
Effort from 20 yards by Gary Harkins. Allan McGregor makes a comfortable save.69:35 Substitution
Substitution Alejandro Bedoya replaces Maurice Edu.66:46
Foul by Dorin Goian on Gary Harkins, free kick awarded. Free kick taken by James Fowler.64:30
Paul Heffernan takes a shot from 12 yards. Save by Allan McGregor.62:36
Liam Kelly fouled by Maurice Edu, the ref awards a free kick. Dean Shiels takes the free kick.61:36
Gary Harkins has an effort at goal from just outside the box which goes wide of the left-hand upright.58:02
Inswinging corner taken left-footed by Sone Aluko from the right by-line.57:49
Sone Aluko takes a shot from 18 yards. Mohamadou Sissoko manages to make a clearance.56:04
Free kick awarded for a foul by Danny Racchi on Kirk Broadfoot. Free kick crossed by Steven Davis, clearance made by Danny Racchi.55:31
The assistant referee signals for offside against Paul Heffernan. Allan McGregor restarts play with the free kick.55:21
Dean Shiels fouled by Lee McCulloch, the ref awards a free kick. Dean Shiels restarts play with the free kick.54:07
Inswinging corner taken right-footed by Dean Shiels, clearance made by Dorin Goian.53:54
Effort on goal by Gary Harkins from 18 yards. Blocked by Steven Whittaker.53:14
The assistant referee signals for offside against Paul Heffernan. Dorin Goian takes the free kick.51:14
Rory McKeown produces a left-footed shot from just outside the penalty box that misses to the left of the goal.49:56
Lee McCulloch fouled by Dean Shiels, the ref awards a free kick. Kirk Broadfoot restarts play with the free kick.47:14
Corner from right by-line taken by Steven Davis.46:03
Liam Kelly concedes a free kick for a foul on Nikica Jelavic. Free kick crossed left-footed by Sone Aluko.45:01
The referee starts the second half.45:00 +2:33 Half time
Half Time The whistle is blown to end the first half.45:00 +2:07
Corner taken right-footed by Dean Shiels, clearance by Nikica Jelavic.45:00 +0:46
Effort from inside the area by Lee McCulloch misses to the right of the goal.44:20
Free kick awarded for a foul by James Fowler on Kirk Broadfoot. Steven Davis crosses the ball in from the free kick, save by Cameron Bell.43:48
Shot by Kane Hemmings from 20 yards. Save made by Cameron Bell.43:03
Free kick awarded for an unfair challenge on Danny Racchi by Kirk Broadfoot. Manuel Pascali takes the direct free kick.41:14
Unfair challenge on Sone Aluko by Mohamadou Sissoko results in a free kick. Kirk Broadfoot takes the free kick.40:34
Corner taken by Dean Shiels from the left by-line, clearance made by Dorin Goian.39:49 Substitution
Substitution Kane Hemmings comes on in place of Kyle Lafferty.35:14
The ball is delivered by Rory McKeown.34:01
Free kick awarded for an unfair challenge on James Fowler by Kyle Lafferty. Cameron Bell takes the free kick.33:01
Free kick awarded for a foul by Mohamadou Sissoko on Sone Aluko. Steven Davis restarts play with the free kick.25:30
Liam Kelly concedes a free kick for a foul on Sone Aluko. Steven Davis takes the direct free kick.24:34
James Fowler gives away a free kick for an unfair challenge on Kyle Lafferty. Steven Whittaker takes the direct free kick.23:24
Steven Davis takes a corner.23:14
Sone Aluko takes a shot. Mohamadou Sissoko gets a block in.22:36
The ball is delivered by Dean Shiels, save made by Allan McGregor.22:21
Dean Shiels crosses the ball, clearance by Kirk Broadfoot.17:36
Ross Perry challenges Paul Heffernan unfairly and gives away a free kick. Direct free kick taken by Dean Shiels.16:52
Gary Harkins produces a cross.15:46
Outswinging corner taken from the right by-line by Steven Davis.15:08
James Fowler concedes a free kick for a foul on Kirk Broadfoot. Direct free kick taken by Kyle Lafferty.14:39
The offside flag is raised against Nikica Jelavic. Free kick taken by Cameron Bell.12:58
Nikica Jelavic takes a shot. Comfortable save by Cameron Bell.10:28
Mohamadou Sissoko challenges Kyle Lafferty unfairly and gives away a free kick. Free kick taken by Kyle Lafferty.9:48
Inswinging corner taken from the left by-line by Dean Shiels.9:08
Inswinging corner taken by Dean Shiels, save by Allan McGregor.6:42 Substitution
Substitution Danny Racchi is brought on as a substitute for Danny Buijs.5:53
Shot from outside the box by Kyle Lafferty goes wide right of the target.5:04
Foul by Maurice Edu on Danny Buijs, free kick awarded. Free kick taken by Liam Kelly.3:09
Sone Aluko delivers the ball.1:33
Nikica Jelavic is flagged offside by the assistant referee. Manuel Pascali takes the direct free kick.1:12
Rory McKeown gives away a free kick for an unfair challenge on Sone Aluko. Direct free kick taken by Steven Whittaker.0:00
The referee gets the game underway.
Live data and text provided by our data suppliers