Jose Fonte's late header earned a point for Southampton and denied Sunderland their first win of the season.
The Black Cats had been on course for victory thanks to Emanuele Giaccherini's second-minute header.
Jay Rodriguez had an early effort ruled out for offside, with replays proving inconclusive, while Saints team-mate Rickie Lambert had several chances.
But in the 88th minute, Fonte headed home James Ward-Prowse's free-kick from close range for a dramatic equaliser.
Sunderland, who were beaten by Fulham on the opening day of the season, took an early lead with Italian Giaccherini heading home Sebastian Larsson's corner from close range.
Southampton responded well and spent the next 45 minutes peppering the visitors' goal.
First Rodriguez had a goal disallowed for offside while Lambert saw his low header kept out at full stretch by Keiren Westwood before heading over from close range later in the half.
Rodriguez then had a prodded effort from 12 yards deflected wide before Saints goalkeeper Artur Boruc had to be alert to keep out Larsson's inswinging corner.
Southampton felt they should have had two penalties in a half they completely dominated with Ward-Prowse adding to his growing reputation with a fine 45 minutes for the south-coast club.
Fonte was dragged down in the area when trying to get his head on Ward-Prowse's free-kick while, just before the break, Adam Lallana was manhandled by Sunderland defender John O'Shea, with referee Lee Mason waving away both appeals.
The hosts introduced record signing Dani Osvaldo and Nathaniel Clyne at the break, with Morgan Schneiderlin and Luke Shaw replaced.
Sunderland also made a change, bringing on Ji Dong-Won for Stephane Sessegnon - and the South Korean almost made an immediate impact, latching onto a long ball before volleying a weak side-foot shot towards goal.
Lallana went close to an equaliser, with his half-volley from Lambert's knockdown flying just past Westwood's far post.
The game continued at a frenetic pace and Sunderland winger Adam Johnson, who had been relatively quiet, came in from his flank to send a shot just over Boruc's crossbar.
Forward Osvaldo's first contribution came just before the hour, with his fine cross headed straight at Westwood by Rodriguez, while at the other end Ji's pass played in Altidore but his shot was smothered by the legs of an on-rushing Boruc.
Southampton continued to press, with Lambert forcing a fine full-stretch save from Westwood with a driven shot from 20 yards and the England international then heading another chance straight at Sunderland's keeper.
Osvaldo was booked for unsporting behaviour late on and was lucky not to see red for a late tackle soon after as it looked as though Sunderland would hold on.
But defender Fonte's late heroics meant Paolo Di Canio's side are still searching for their first victory while Saints remain unbeaten after a 1-0 win at West Brom last weekend.
Southampton manager Mauriccio Pochettino: "We were asleep in the first three minutes [for Emanuele Giaccherini's goal] - it is not acceptable and I take responsibility for it.
"If you look at the chances we created we deserved to win the match but games are not won by chances made but goals scored. I am a bit disappointed because our goal was to get three points. When we review it we will see we deserved to win it."
Sunderland manager Paolo Di Canio:
"We were leading at 1-0 but another set-play cost us two points. It was a cheap free-kick but 1-0 is more than acceptable. We have to be honest, Southampton deserved to win this game because they played better.
"This is a tough league, we all know that, but we have time to improve. However, one point away from home against a fantastic side is a good result."