Tunisia's Youssef Msakni settled a drab North African derby against Algeria when curling home in stoppage time to earn a crucial win in Rustenburg.
The Carthage Eagles join Ivory Coast, who beat Togo 2-1 earlier, at the top of Group D after a barely-deserved win.
Algeria started the brighter and were unlucky not to lead after 29 minutes when Islam Slimani struck the bar.
Hamdi Harbaoui had seemed to waste Tunisia's best chance only for Msakni to settle affairs with a classy finish.
Picking up the ball some 30 yards from goal, the winger cut inside on to his right foot before curling a beautiful strike into the top-right hand corner which left Algerian goalkeeper Rais Mbolhi with no chance.
HOPING FOR MORE
The last time Tunisia were paired in a Nations Cup group with Ivory Coast, they reached the final - as South Africa hosted the competition on its only previous occasion 16 years ago
It was a moment of rare quality to delight the fans inside the Royal Bafokeng Stadium after the previous 90 minutes had been littered with fouls and misplaced passes.
Despite both sides' long history in the competition, the first Africa Cup of Nations meeting between the neighbours proved to be a more nervy affair than the heated encounter some had keenly anticipated.
Tunisia encountered a difficult start to the game as record scorer Issam Jemaa was forced off with a knee problem after coming off worst in a 50-50 challenge with Algerian defender Esseid Belkalem.
As his replacement Harbaoui took time to settle, Algeria took control of the opening period - with Marseille's Foued Kadir wasting a half-chance after 23 minutes.
Three minutes later, the 1990 champions produced their best move of the match as highly-rated Valencia midfielder Sofiane Feghouli swung in a fine left-wing cross, which Slimani headed against the bar with Tunisia keeper Moez Ben Cherifia beaten.
Algeria have now failed to score in five of their last seven matches at the Africa Cup of Nations
Tunisia's sole opportunity of the half came when Saber Khalifa struck an angled drive straight at Mbolhi, with the former Manchester United triallist making a comfortable block.
The half-time introduction of playmaker Oussama Darragi by Tunisia coach Sami Trabelsi turned the 2004 champions into a more potent attacking unit, but it was still Algeria who created the better chances.
Midfielder Adlene Guedioura, who plays for Championship side Nottingham Forest, hit a dipping 25-yard drive which flew inches over while Djamel Mesbah also narrowly missed with a long-range effort.
After 75 minutes, Darragi carved open the Algerian defence with a fine pass but the recipient Harbaoui failed to dig the ball out from under his feet and dragged his shot wide with only Ben Mustapha to beat.
Algeria thought they had won a penalty when Feghouli tumbled in the box but the 1990 champions came even closer to scoring four minutes from time as Tunisia defender Aymen Abdennour inadvertently deflected a driven cross off the knee of Kadir and harmlessly over the goalline.
The sucker punch came minutes later as Qatar-based Msakni, who scored twice at last year's Nations Cup, displayed the qualities that have seen his name constantly linked with French giants Paris Saint Germain.
Tunisia will now look ahead to Saturday's encounter against Didier Drogba's Ivory Coast with increased confidence, while Algeria coach Vahid Halilhodzic knows his side must avoid defeat against Togo.
The final Group D encounter of the day means that all 16 teams at the 2013 Africa Cup of Nations have now been in action, with no outstanding team have emerged so far.
Tunisia coach Sami Trabelsi:
"You have to be really patient in a match like that as they are often decided at the last minute. Unfortunately, when Issam Jemaa came off early on, it left us missing something up front.
"Bravo to Algeria - they played a good match. Sometimes there is no logic in football and a match can change any time, but we always believed we could win."
Algeria coach Vahid Halilhodzic:
"The result was an enormous disappointment. We deserved to get something out of the game. I can't criticise any of my players.
"It's cruel, but that's football. My team largely dominated, but I told them Msakni could cause trouble with his right foot and we were a little naive at the moment that decided the game."