Sone Aluko's stunning volley gave Hull victory over Newcastle in a highly entertaining encounter.
Loic Remy gave the hosts the lead when he headed home from Papiss Cisse's miscued shot at the back post.
Robbie Brady levelled with a low effort before Remy scored his second when Yohan Cabaye's shot fell into his path.
Brady's free-kick was glanced in by Ahmed Elmohamady shortly after the restart and Aluko won it with a superb finish from 18 yards.
After a slow start to the season - the Magpies did not manage a shot on target in their first two outings - Alan Pardew's side came into this match on the back of successive league wins over Fulham and
Newcastle played in third gear - Pardew
Hull have made a steady start to their first season back in the top flight after three years in the Championship, having picked up four points from their opening four games.
The hosts began the game in confident mood and went ahead in the 10th minute when Moussa Sissokho's cross to the back post was hit into the ground by Cisse. His attempted shot turned into the perfect ball for Remy to head home.
The goal was deserved reward for Pardew's side, who made a bright start to the match as they looked to secure a fourth successive win in all competitions.
However, they started to allow errors into their game and, after goalkeeper Tim Krul did brilliantly to deny Danny Graham from a Brady cross, they conceded their first goal at St James' Park this season.
Conor McNamaraBBC Match of the Day commentator
Hull City have shown today that they can mix it with the big boys. Sone Aluko's magical winner has gave them all three points at Newcastle for a first away win of the season.
Newcastle failed to clear following the Dutchman's stop and, with the defence out of position, Aluko threaded a neat pass to the excellent Brady, who struck a low shot through the legs of Krul for his third goal of the season.
Newcastle went back in front against the run of play just before the break in fortuitous circumstances. Cabaye's shot from 25 yards hit Hull defender James Chester and Remy confidently slotted home his second.
The goal was harsh on the visitors but they responded well and levelled once more.
Aluko was fouled by Vurnon Anita and, from the resulting free-kick, Brady found the unmarked Elmohamady, who saw his header go in off the post.
With Remy tiring and Cabaye forced off after a heavy challenge from Curtis Davies the hosts began to run out of ideas and they fell behind to Aluko's brilliance.
From George Boyd's cross, Nigerian Aluko superbly guiding his effort in off the same post as Elmohamady's header.
As the game entered five minutes of injury-time Remy had the chance to score his third but he fired wide from six yards out and Hull held out for a first away win of the season.
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.