A second-half goal from Dave MacKay gave St Johnstone a shock first win at Celtic Park since 1998.
Kris Commons failed to convert from the penalty spot after Saints keeper Peter Enckelman brought down Anthony Stokes in the box in the third minute.
But St Johnstone eventually broke the deadlock in the 60th minute, with MacKay's shot deflected into the net.
Celtic's first league defeat means that no Scottish Premier League club has a 100% record.
The first half was characterised by sustained Celtic pressure, but an inability to convert chances.
Saints' keeper Enckelman handed Neil Lennon's side the first of those chances, when he was judged to have brought down Stokes inside the box.
But the Finn, who escaped a booking, soon made amends, diving to his right to save the resulting penalty kick from Commons.
That missed opportunity sparked Celtic into life, and Lennon's side enjoyed a spate of first half chances, most created by the industry of Shaun Maloney who set up chances for Commons and Stokes, as well missing an opportunity created by Adam Matthews in the 26th minute.
But the best chance for the home side came from a set piece, Commons almost making amends for his penalty miss with a terrific curling free kick which came off Enckelman's post five minutes before the break.
And Celtic seemed to approach the restart with gusto, Commons ekeing out another chance seconds into the half.
But it was St Johnstone who took a shock lead on the hour mark.
A move that began with a Jody Morris drive, which was beaten away by Fraser Forster, then saw MacKay volley towards goal from the right hand side of the box, his shot deflecting off Joe Ledley and past Forster.
After dominating possession throughout the match, that sucker-punch appeared to stun the Parkhead side, who struggled to regain their tempo.
Substitute Paddy McCourt, who replaced Commons immediately after the goal, offered a piece of individual skill almost immediately, twisting and turning his way through a wall of blue jerseys in the St Johnstone box before eventually being dispossessed.
In defence though, Celtic were in disarray, confusion between Daniel Majstorovic and Forster almost allowing Saints substitute Sean Higgins clear through on goal 20 minutes from the end.
McCourt continued to create chances, a shot five minutes from time coming off the post, the Irishman drawing a free-kick on the edge of the box out of Saints defender Callum Davidson, but Maloney's set piece was marginally off target.
And a hard-working and resolute St Johnstone defence managed to neutralise a flurry of late chances to secure Derek McInnes's first win of the season and increase the pressure on Celtic ahead of their midweek Europa League tie.