It was the visitors who started the brighter of the two sides and Scott Sinclair could have given them the lead with less than two minutes gone.
Win helps Pulis shrug off selection debate
A pinpoint ball from Nathan Dyer was met by Sinclair but he side-footed his effort well over the bar.
At the other end, the first chance for Stoke fell to Crouch from a corner but he headed wide from close range.
Swansea were keeping the ball well and passing it around with ease, but the Stoke threat was always present from long balls.
And so it proved as the visitors failed to learn from their earlier let-off and Upson was on hand to head home his first goal for the Potters.
Upson was then in more familiar territory just minutes later, putting in an excellent sliding tackle to prevent Swansea from getting back on level terms.
Did you know?
Swansea have picked up just nine points away from home this season - one better than Fulham
They have kept just one clean sheet in their last 12 matches in all competitions
Neil Taylor went on a tremendous run that took him past four men and into the area but, as he was about to pull the trigger, Upson slid in and deflected the ball behind for a corner.
Stoke soon doubled their lead through Crouch, who headed home a long throw from Shotton for his second goal in as many games.
Tremmel will feel he should have done better as he fumbled the ball over the line.
Another long throw almost saw Stoke add a third goal. The ball was headed on at the near post by Crouch and fell for Jon Walters four yards out but his header went over the bar.
The visitors came within inches of registering a goal with just minutes remaining but Asmir Begovic tipped Caulker's effort on to the crossbar.
The win moves Stoke up to 12th in the table, ending a run of four Premier League defeats, while Swansea's problems on the road resurfaced - Brendan Rodgers's men have only picked up nine points from 12 away trips this season.
Rodgers not blaming keeper switch
Stoke City manager Tony Pulis:
"We worked very hard out there. I thought the lads were fantastic. It has been a difficult season for us, today was our 43rd game and we have been all over Europe and are still in the FA Cup.
"It has been hard for the players and at times they have looked tired but I thought their energy levels today were terrific, I don't know where they get them from."
Swansea boss Brendan Rodgers:
"I got a call from the physio this morning and he told me about Michel [Vorm] who was too weak to play, which was obviously a big blow for us but I thought that Gerhard Tremmel came in and did very well considering that he has not played many games.
"We should have defended the two goals better, we have worked on that during the week. We are disappointed with the goals but there were parts of our play that were superb."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.