Warrenpoint moved four points clear of the Premiership basement thanks to a Daniel Hughes double at Stangmore Park.
Matt Hazley's low strike gave the Swifts an early lead but Hughes brought Town level with an excellent lob over keeper Gareth Buchanan on 36 minutes.
Warrenpoint boss Barry Gray was sent from the dugout after remonstrating before Swifts striker James Costello headed wide from five yards.
Hughes won it seven minutes from time with a classy shot off the post.
The visitors were caught napping in the second minute when Ryan Harpur picked out the unmarked Hazley from a free-kick and the ball was clinically despatched into the bottom corner.
It became a scrappy affair in a strong which favoured Dungannon in the first half.
Timmy Grant's pass put Hughes through for the equaliser, with the striker lifting the ball over Buchanan and into the net from a tight angle.
Town keeper Jonathan Parr punched clear Darren Boyce's powerful effort at the start of the second half.
Gray was ordered to the stands midway through the second half as he reacted to a touchline tackle.
The game turned on two chances, one squandered and the other taken.
Costello's glaring miss was quickly followed by the winner, with Hughes curling the ball home via the woodwork.
"We were pressing and I just turned, it opened up and I sent it into the corner," said a delighted Hughes after the game.
"We did really well and thoroughly deserved the victory although if Costello scored it would have been a different story.
"It's vital that we stay up - we have the ability in the squad to do that and the move back to our Milltown ground will definitely help."