Match ends, Rotherham United 1, Rochdale 2.
Peter Vincenti's winner at Rotherham put Rochdale into the FA Cup third round for the first time since 2005.
Lee Frecklington seized on a loose pass and slotted past Josh Lillis to put the League One side ahead early on.
Kieran Agard was twice denied by goalkeeper Lillis before Rochdale, fifth in League Two, recovered.
Matty Lund turned in Michael Rose's cross to level and the Millers had Michael Tidser sent off before sub Vincenti converted Joe Bunney's centre.
Tidser received his second yellow card 16 minutes from time and Rochdale, who had improved in the second period, were able to make the advantage of an extra man count.
Home keeper Adam Collin kept out efforts from Scott Hogan, Jack O'Connell and Rose either side of the midfielder's dismissal.
Vincenti had been brought on as a first-half substitute for centre-back Oliver Lancashire as visiting manager Keith Hill changed formation following a poor start to the match.
Ben Pringle curled a late free-kick wide as 10-man Rotherham tried in vain to earn a replay.
VIEW FROM THE DRESSING ROOM
Rotherham boss Steve Evans told BBC Radio Sheffield:
"Obviously I'm upset. I don't think we got going in any stage of the match but despite that we got in front.
"In the second half we were trying to find a bit of play and struggling.
"The game-changer moment to go out of the cup was down to Michael Tidser getting sent off."
Rochdale manager Keith Hill told BBC Radio Manchester:
"It was a very good response to a very bad first 25 minutes of the game. We could have lost the game in that first 25 minutes and that would probably have been down to ourselves.
"Oliver [Lancashire] has been unbelievable for me, and he'll be a brilliant player for us, but I had to make the change early.
"Our game plan didn't work, so I had to change it. And I changed it so that we could win the game.
"This wasn't about the league - this was about me trying to earn money for the football club. I thought we were business-like and I made a business-like decision in the early stages."
- 8FrecklingtonBooked at 80mins
- 20MorganBooked at 55mins
- 2BrindleyBooked at 29minsSubstituted forDavisat 76'minutes
- 23TidserBooked at 74mins
- 3BennettBooked at 90mins
- 6LancashireBooked at 11minsSubstituted forVincentiat 22'minutes
- 12LundBooked at 86mins
- 11HoganSubstituted forAllenat 86'minutes
- 16DoneSubstituted forBunneyat 78'minutes
- Graham Scott
- Shots on Target
Second Half ends, Rotherham United 1, Rochdale 2.
Foul by Mark Bradley (Rotherham United).
Ian Henderson (Rochdale) wins a free kick on the right wing.
Rhys Bennett (Rochdale) is shown the yellow card.
Attempt missed. Ben Pringle (Rotherham United) right footed shot from outside the box is close, but misses to the left from a direct free kick.
Alex Revell (Rotherham United) wins a free kick in the attacking half.
Foul by Peter Vincenti (Rochdale).
Substitution, Rochdale. Jamie Allen replaces Scott Hogan.
Matthew Lund (Rochdale) is shown the yellow card.
Foul by Alex Revell (Rotherham United).
Rhys Bennett (Rochdale) wins a free kick in the attacking half.
Goal! Rotherham United 1, Rochdale 2. Peter Vincenti (Rochdale) right footed shot from very close range to the bottom left corner. Assisted by Joe Bunney.
Lee Frecklington (Rotherham United) is shown the yellow card.
Substitution, Rochdale. Joe Bunney replaces Matt Done.
Alex Revell (Rotherham United) wins a free kick on the left wing.
Foul by Rhys Bennett (Rochdale).
Attempt blocked. Lee Frecklington (Rotherham United) right footed shot from the left side of the box is blocked.
Substitution, Rotherham United. Claude Davis replaces Richard Brindley.
Second yellow card to Michael Tidser (Rotherham United).