David Cox struck in the second half at Somerset Park to book Annan a place in the Ramsdens Cup semi-finals.
After a slow start Annan began to take control and Cox was denied with two efforts and Scott Gibson could not direct a header on target.
But after 58 minutes Cox turned in Sean O'Connor's cross from close range to put the Third Division side ahead.
Mark Roberts headed wide and Ryan Connolly had a header saved by Craig Summersgill as Ayr sought an equaliser.
Ayr: Cuthbert, Malone, Ross Robertson, John Robertson, Smith, Tiffoney (McGowan 59), Trouten, Connolly, Burke (Moffat 54), Armstrong, Paterson (Roberts 61).Subs Not Used: McWilliams, Crawford.
Booked: Ross Robertson, Armstrong, Trouten.
Annan Athletic: Summersgill, Peter Watson, Gibson, McGowan, Muirhead, McKechnie, Jardine (Steele 81), Sloan, Harty, O'Connor (Bell 61), Cox (Gilfillan 81).Subs Not Used: Hyslop, MacBeth.
Goals: Cox 58.
Referee: Craig Charleston