Andy Little rescued a point for Rangers as they began life in Scottish Division Three with a draw at Peterhead.
Barrie McKay fired the visitors ahead midway through the first half.
Rory McAllister's composed second-half finish levelled matters and Scott McLaughlin's powerful strike edged the hosts ahead eight minutes from the end.
But the scores were all square again in the final minute when Little forced the ball over the line after Kevin Kyle's header had struck the bar.
The draw was certainly not the result the 4,500-strong capacity crowd were expecting as title favourites Rangers were given a rude awakening in the Scottish Football League's lowest division.
Rangers dominated possession early on, but it was the hosts who threatened first, McAllister's flicked header at a corner being cleared off the line by McKay.
McAllister should have done better than scoop the ball into Alexander's arms after getting the break of the ball in the box moments later.
It was from a simple long ball that Ally McCoist's side took the lead as Dorin Goian's pass was nodded down by Lee McCulloch to McKay, who calmly slotted past Paul Jarvie.
McAllister's frustration in front of goal continued as he skewed his latest opportunity across the face of goal after the Rangers defence were caught cold at a free kick.
The striker's curling shot was tipped onto the bar and out by Neil Alexander as Rangers continued to look sluggish in the second half.
And the former Inverness player did got his name on the scoresheet with arguably his most difficult chance of the match, turning away from Carlos Bocanegra in the box and calmly swerving a shot past Alexander.
Francisco Sandaza replaced McKay as Rangers looked to re-assert themselves and the Spaniard forced three saves from Jarvie, who also blocked a rebound shot by Dean Shiels after the third Sandaza effort.
McCulloch was next to test Jarvie with a stinging volley that the keeper punched clear.
However, Rangers then failed to deal with Ryan Strachan's corner and McLaughlin lashed the ball home with a rising shot from the edge of the box.
McCoist's side went back on the attack and substitute Kyle's header from Shiels' corner struck the underside of the bar only to be nudged over the line by Little on the rebound.
Peterhead: Jarvie, Scott Ross, Strachan, MacDonald, Sharp,Redman, Cowie (Deasley 87), McLaughlin, Noble, Winters (Bavidge 58), McAllister (Maguire 83). Subs Not Used: McCallum, McBain.
Booked: MacDonald, Sharp, Cowie.
Goals: McAllister 64, McLaughlin 82.
Rangers: Alexander, Broadfoot, Goian, Bocanegra, Little, Black, Macleod (Kyle 82), Wallace, McCulloch, Shiels, McKay (Sandaza 65). Subs Not Used: Gallacher, Crawford, Hutton.
Goals: McKay 27, Little 90.
Ref: Stephen Finnie