Michael Moffat was 10-man Ayr's matchwinner, as his stoppage-time goal cut Stranraer adrift at the foot of the Second Division table.
The hosts were inches away from taking the lead after only 11 minutes when Mark Roberts' effort hit the post.
However, the visitors took the lead when Michael Moore struck from close range after half an hour.
Moffat's penalty levelled the scores however, after Grant Gallagher upended Roberts.
Marc Twaddle struck the crossbar as the Honest Men mounted second-half pressure, before Ryan McStay was dismissed for violent conduct to give the away side a numerical advantage.
But the visitors were unable to hold onto their point as Moffat's injury-time finish earned a late victory.
Brown, Hunter, McAusland, McCann, Twaddle, Crawford, McStay, Sinclair, Moffat, Roberts (Winters 90), Mackinnon (Shankland 80). Subs Not Used: William Muir, Ross Robertson, Wyllie.
Hunter, Moffat, Twaddle, Roberts.
Moffat 57 pen, 90.
Mitchell, Staunton, McKeown, Dunlop, MacGregor, Wright, Winter, Gallagher, Love, Moore (One 85), Malcolm. Subs Not Used: Cadwell, Lloyd Kinnaird, Agnew, Rafferty.