Brian Noble won his first Super League game as Salford boss as they saw off Castleford to move off bottom spot.
Nathan Massey, Justin Carney and Adam Milner put Tigers 14-6 up, despite Matty Ashurst's try for Salford.
But Danny Williams, Andrew Dixon and Vic Mauro scored for the hosts before Oliver Holmes' try reduced the deficit.
Theo Fages extended Salford's lead, and after Jamie Ellis and Jordan Thompson put the Tigers ahead, Ryan McGoldrick got the Reds' winning try.
McGoldrick's try, converted by Marc Sneyd for his fifth goal, brought to an end a thrilling encounter at the Salford City Stadium, which sees Noble's new charges leapfrog Castleford at the bottom of the Super League Table.
However, the game did not begin as planned for Noble, as Massey charged through numerous tackles to put the visitors in front after just four minutes.
And despite Ashurst evading a challenge from Michael Shenton with Sneyd converting to put Salford 6-4 up, Carney ducked inside from the line to cross and Milner went over to restore Tigers' lead.
Salford were not deterred, and Williams's score - given after video referral - and Andrew Dixon's try gave the hosts a 16-14 half-time lead.
Almost immediately after the break Mauro crossed for his first league try of the season before Frenchman Fages bundled over after Oliver Holmes had crossed for Tigers.
Once more the video referee was used to award a try when Rangi Chase's grubber kick was put down by Ellis to close the gap before Thompson dived over after fine work from Shenton to give the visitors a 30-28 lead.
But Chase was at fault for Salford's winner, with McGoldrick intercepting his pass before running 40 metres to touch down and give the hosts victory.
VIEW FROM THE DRESSING ROOM
Salford head coach Brian Noble:
"I'm really pleased for the players. We finished on the right side of a basketball-style game and it was thrilling and exciting.
"We had to show plenty of character to defend like we did in the last five minutes.
"We did a lot of good things going forward and I know there is work to do after looking at the league ladder. But it can be cured in time and we need to start raising the bar."
Castleford caretaker boss Danny Orr:
"We didn't deserve to win and to concede 34 points was disappointing.
"We were outmuscled and our kicking game was terrible in the first half.
"We did not perform and self-destructed."
Salford: Sneyd; Broughton, Gibson, Gleeson, D Williams; Fages, McGoldrick; Griffin, Godwin, Emmitt, Ashurst, Dixon, Wild.
Replacements: Foran, Mauro, James, A Walne.
Castleford: Tansey; Carney, Shenton, Thompson, Dixon; Chase, Ellis; Mason, Milner, Huby, Hauraki, Holmes, Massey.
Replacements: Walker, Clark, Gilmour, Boyle.
Referee: Thierry Alibert (RFL).