Paul Scharner's late strike handed West Brom victory as Newcastle failed to win for the sixth successive league match.
Peter Odemwingie gave West Brom the lead when he fired in from 16 yards.
Demba Ba equalised with a superb 20-yard free-kick before the Baggies again went ahead when Gareth McAuley nodded in from close range.
Ba made it 2-2 with a firm shot, but Scharner's volley ensured that West Brom won at Newcastle in the league for the first time since 1977.
It was heartbreaking for the Magpies who, having twice fought back and having created several chances, were dealt their second home defeat this season and their fourth loss in the last six matches.
And a stat the owners would prefer to gloss over, is that they have not won at the Sports Direct Arena since the ground's name was changed from St James' Park on 9 November.
The only stat West Brom were interested in was securing their second successive win - one that was built on great defensive work and fuss-free counter-attacking.
Both sides, however, deserved pats on their backs after delivering a thoroughly entertaining spectacle.
Midfielder Chris Brunt and striker Peter Odemwingie were the big threats in attack for the Baggies while, once again, Senegal striker Ba was the star man for Alan Pardew's team. All three were key to the goals in the first half.
The Baggies pair combined for the opener, when Brunt threaded a pass through to Odemwingie with the outside of his foot. The Nigerian looked up before drilling his shot past Tim Krul.
Ba, who scored his 12th and 13th of the campaign, then levelled for Newcastle when he took responsibility for an attacking free-kick, in the absence of Ryan Taylor, and curled his shot firmly past Ben Foster.
But, just before half-time, the Baggies again seized the advantage when playmaker Brunt swung a free-kick to the far post, which was headed back across the six-yard area by Jonas Olsson and nodded into the net by McAuley.
West Brom spent much of the second half trying to protect their lead under severe pressure from the Magpies.
Slovenian Haris Vuckic, 19, impressed on his full league debut. He hit the bar and forced a greeat save from Foster, while Ba went close to scoring what would have been the 20,000th Premier League goal with a 20-yard strike that flew a fraction wide.
He missed that landmark as Marc Albrighton of Aston Villa scored first in their match with Arsenal.
But Ba brought Newcastle level again when Hatem Ben Arfa teed up his team-mate to fire high past Foster from just inside the area.
West Brom's best chance during that early period of the second half fell to Shane Long, who rounded Krul but struck the bar with his shot.
But, with five minutes of normal time remaining, the Baggies grabbed what proved to be the winner. Long's looping cross from the right found Odemwingie who headed back into the path of Scharner. The Austrian stretched out a leg and volleyed in.
West Brom manager Roy Hodgson:
"It was remarkable to come back again and get the third goal. We squandered good opportunities in the first half, although they weren't clear-cut.
"We were solid defensively and worked very, very hard. If Shane [Long] had not hit the crossbar then life would have been more comfortable.
"It's been misfortune more than anything as to why we have not won as many points at home as we have away."
Newcastle manager Alan Pardew:
"You would have expected us to win after we got back to 2-2. They had a bit of fortune for their third goal.
"We were cheap with possession at times.
"Maybe the win is playing on our minds - we want that under our belts. We were desperate at times. We have to be more patient and more controlled. We were throwing numbers forward and got caught at times."