Dean Shiels struck deep into extra time at Hampden to break stubborn resistance from Ayr United and send Kilmarnock into the League Cup final.
Ayr keeper Kevin Cuthbert made a string of fine saves in the second half.
But he was finally beaten after 109 minutes at Hampden, with Shiels powering home a shot after Heffernan's low drive was spilled.
Ayr's defensive approach made for a one-sided contest, with Kilmarnock enjoying almost 70% of possession.
The Rugby Park side will meet either Celtic or Falkirk, who meet on Sunday, in the final in March.
The intense rivalry between the fans was not mirrored on the pitch as the match opened with a whimper.
Some 23 minutes had passed without incident before Ayr United defender John Robertson headed a Garry Hay delivery away from underneath his own crossbar.
Interview - Kilmarnock forward Dean Shiels
Hay then tried his luck from long range but Cuthbert held the midfielder's shot comfortably.
Michael McGowan curled a free kick too high after Liam Kelly had brought a rare Ayr United counter-attack to a halt with a clumsy tackle on the edge of his penalty box.
A cute back heel from Paul Heffernan opened up the Ayr defence but Dean Shiels could not steer his shot on target.
Heffernan then picked out Ben Gordon with a great pass and the full-back charged into the penalty box only to shoot straight at Cuthbert, who was guarding his near post.
James Dayton had Killie supporters on their feet with an enterprising run early in the second half but, having waltzed down the byeline, his cutback failed to find a team-mate.
A Heffernan shot was deflected wide and Shiels slammed a promising free kick into the defensive wall after the Nothern Irishman had been cynically tripped by McGowan.
Cuthbert then made routine saves to deny Gordon and Hay as the Scottish Premier League side continued to dominate.
On 66 minutes, Cuthbert made an impressive double save to keep the score level.
The Ayr keeper was at full stretch to stop an arrowing drive from Dayton and stuck out a foot to thwart Gary Harkins as the midfielder latched on to the rebound.
There were more heroics from Cuthbert as he tipped over a rocket from Shiels then scrambled away a Manuel Pascali header from the resultant corner.
Interview - Ayr United manager Brian Reid
The First Division outfit registered their first shot on goal after 82 minutes, with Cammy Bell doing well to gather a smart, curling effort from Alan Geggan.
Killie responded with a rising strike from Hay but Cuthbert was again equal to it, his fingertips diverting the ball over the crossbar.
The pattern of play continued in extra time, with Ayr sitting deep in their own half and Kilmarnock probing for an opening.
The breakthrough came in the second period as Cuthbert got down to stop raking shots from Shiels and Heffernan but he could not keep out the third effort as Shiels thumped the ball home from 12 yards.
Ayr United 01 Cuthbert 03 Malone 04 Robertson 05 Smith 02 Tiffoney 06 Geggan Booked 07 Trouten (Tomsett - 113' ) 08 McKernan (Dodd - 75' ) 11 McGowan Booked 09 Moffat Booked 10 Wardlaw (Roberts - 60' ) Substitutes 20 McWilliams 16 Robertson 14 Dodd 15 Tomsett 12 Roberts Kilmarnock 01 Bell 03 Hay Booked (Buijs - 105' ) 04 Fowler 06 Nelson 19 Gordon 08 Kelly Booked 10 Dayton Booked (Silva - 79' ) 11 Harkins (Racchi - 89' ) 20 Shiels 29 Pascali 14 Heffernan Substitutes 17 Letheren 13 Kroca 07 Silva 09 Buijs 22 Racchi