Celtic paid the price for missed chances as Hibs twice pegged them back.
The home side were utterly dominant in the first half but had only a Mikael Lustig strike to show for it, with Gary Hooper twice hitting the woodwork.
Hibs were a different side in the second half and levelled as Tim Clancy punished a Lustig error for a tap-in.
Lustig forced in a second via Hibs' James McPake after a Ben Williams spill. But a great individual goal by Paul Cairney earned Hibs a point.
The visitors had started the match pretty steadily but nonetheless found themselves behind after just 11 minutes.
James McPake made a crucial intervention to deny Victor Wanyama a tap-in from a Filip Twardzik corner, but no-one in the visiting defence could prevent Lustig's thunderous effort from 12 yards from Twardzik's subsequent corner out the net.
Celtic assumed complete control of the match and came within inches of a second when Hooper picked up possession on the edge of the box only to see his powerful toe-poke come off the underside of the bar.
Williams in the Hibernian goal was seeing plenty of the ball and he reacted brilliantly to push away a powerful Wanyama drive from 10 yards.
Forrest was next to try his luck, this time from outside the penalty area, but his low strike whistled just past Williams' right-hand post.
The game was in danger of drifting into non-event territory as half-time approached, though Hibs briefly threatened when McPake's header across goal allowed Leigh Griffiths to try an adventurous overhead-kick, which flew well off-target.
Celtic sprang into life and first Watt chipped wide when he should have scored before Hooper was again denied by the frame of the goal, heading against Williams' left-hand post from Adam Matthews' cross.
Whatever Pat Fenlon said to his players at the interval certainly had the desired effect, though.
Within two minutes of the restart, a Griffiths free kick from the right was knocked against the post by Eoin Doyle.
But the equaliser did arrive moments later after woeful defending by Lustig.
The Swede attempted to guide the ball back to Forster but failed miserably to do so, allowing Clancy to nip in, round the keeper and tap in to an empty net.
An inspired Paddy McCourt run and shot came close to restoring Celtic's lead and though Hibs were playing a far greater role in the match, they fell behind again when Williams spilled a long-range Twardzik free kick, allowing Lustig to slide in and, despite the best efforts of McPake on the line, force the ball over the line.
The home crowd were stunned again, though, when Cairney showed superb skill to get past two challenges and then slot the ball past Forster.
Celtic continued to create chances as the game wore on, and Twardzik passed up a glorious opportunity with eight minutes left.
A Kris Commons free kick was punched into the Czech's path by Williams but he blasted over the bar from eight yards.
McPake was playing heroically at the back for Hibs and he did just enough to deflect a Commons shot wide as time ticked away.
And in the end, Hibs held on comfortably to continue their impressive start to the league season, though Celtic have only themselves to blame for dropping two points.