One of those punished was in-form keeper Wes Foderingham and it was his replacement Smith who was at fault for the first goal.
Charlie Lee's long throw found Kedwell, and the striker nodded the ball over Smith, who had found himself stranded off his line.
Swindon looked in disarray and boos from the Town fans rang out when top scorer Alan Connell, playing as a lone striker, was brought off after just 39 minutes to be replaced by Ronan Murray.
Did you know?
Swindon had not lost back-to-back league games since August
The Gills grew in confidence and shortly before half-time they missed two opportunities to increase their lead - Lewis Montrose's stunning strike hit the crossbar while Whelpdale saw his header go wide.
The hosts started the second half in a similar fashion and Whelpdale's delicate chip gave them the second goal they deserved.
Swindon rarely tested Gills keeper Paulo Gazzaniga but McCormack's low drive gave the visitors a lifeline.
However, their defeat was confirmed when Whelpdale struck from the spot, after Curtis Weston was brought down by Jonathan Smith.
But with results from other grounds going their way, Di Canio was able to hold his red and white scarf aloft in front of 1,400 travelling fans and celebrate Swindon's promotion in his first season in charge at the County Ground.
Gillingham also strengthened their position in the promotion push and now sit just four points off Crewe, who currently occupy the final play-off place.
Gillingham 3-1 Swindon
VIEWS FROM THE DRESSING ROOM
Swindon Town manager Paolo di Canio told BBC Points West:
"I'm never going to say why, but a month ago I nearly resigned my contract. I have to tell people now that we are up.
"I didn't do this because I promised my fans at the beginning of the season that we would go up.
"If I went now, this team - without Paolo di Canio - are capable of losing 13 games in a row. They've shown that in the last two games.
"I've told them already in the dressing room that we have to change the direction because if you become arrogant like we have, then we are average."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.