Duffy then equalised when he converted from the spot after Curtis Nelson brought down Elliott in the box.
Plymouth's Jamie Griffiths saw red for a dangerous tackle on Luke Garbutt, before Duffy fired in from close range.
VIEWS FROM THE DRESSING ROOM
Cheltenham Town boss Mark Yates told BBC Radio Gloucestershire:
"It was an important win to come back from defeat in midweek and show a bit of character.
"I did feel sorry for them if I'm honest, the result is harsh on them.
"I asked my players to be ruthless and I'm glad we were. It was ugly but you have to show all sides of your game if you are going to be successful."
Plymouth boss Carl Fletcher told BBC Radio Devon:
"It's disappointing. I've got lads in there aged 17/18 years old on the floor who've worked their socks off.
"He [Jamie Griffiths] has worked his socks off today, he's not a striker and he's worked really hard and he's in tears in the changing room disappointed. He's only a young lad and he's come here and wants to do well for the team.
"I can't ask any more from the lads. Today there's been good things and bad things we will take out of it. They're young lads all the way through and they are learning every game"
Booking The referee shows Steve Elliott a yellow card for unsporting behaviour.
Booking Durrell Berry is shown a yellow card.
Unfair challenge on Steve Elliott by Curtis Nelson results in a Penalty.
Russell Penn has an effort at goal from 18 yards. Outswinging corner taken right-footed by Marlon Pack from the right by-line, clearance made by Conor Hourihane. Corner taken left-footed by Luke Garbutt from the left by-line, clearance made by Curtis Nelson.
Isaac Vassell takes a shot. Save made by Jack Butland.
Sido Jombati sends in a cross, clearance made by Durrell Berry.
Substitution Luke Summerfield leaves the field to be replaced by Brian Smikle.
Booking The referee shows Onismor Bhasera a yellow card for time wasting.
Conor Hourihane takes a shot. Save made by Jack Butland.
Substitution Isaac Vassell on for Jared Sims.
Substitution Romain Larrieu replaces Jake Cole.
A cross is delivered by Bagasan Graham, clearance by Jack Butland. Outswinging corner taken by Luke Garbutt. The ball is sent over by Marlon Pack, clearance by Durrell Berry.
The ball is sent over by Bagasan Graham, clearance by Luke Young. Corner from the left by-line taken by Luke Garbutt, The ball is sent over by Marlon Pack, Jake Cole makes a clearance. Alan Bennett takes a shot.
Luke Young takes a shot. Steve Elliott gets a block in.
Foul by Conor Hourihane on Russell Penn, free kick awarded. Marlon Pack takes the free kick.
The ball is delivered by Sido Jombati, Ben Gibson makes a clearance.
Substitution Russell Penn is brought on as a substitute for Joshua Low.
Substitution Bagasan Graham is brought on as a substitute for Kaid Mohamed.
Unfair challenge on Alan Bennett by Jamie Griffiths results in a free kick. Free kick taken by Luke Garbutt.
Outswinging corner taken from the left by-line by Luke Garbutt, clearance made by Curtis Nelson.
Centre by Marlon Pack, Luke Young manages to make a clearance.
Jeff Goulding takes a shot. Save made by Jake Cole.
The offside flag is raised against Darryl Duffy. Indirect free kick taken by Jake Cole.
Handball by Ben Gibson. Luke Garbutt takes the direct free kick.
Assist on the goal came from Jared Sims. Save by Jack Butland.
25:47 Goal scored
Goal - Steve Elliott - Cheltenham 0 - 1 Plymouth An unfortunate own goal is conceded by Steve Elliott. Cheltenham 0-1 Plymouth.
The ball is delivered by Simon Walton, save made by Jack Butland. Short corner taken by Luke Young from the left by-line. Jared Sims takes a shot.
The referee gets the second half started.
45:00 +1:01 Half time
Half Time The first half comes to an end.
Sido Jombati delivers the ball.
Luke Young takes a shot.
Outswinging corner taken by Marlon Pack from the right by-line, clearance made by Jake Cole.
The referee blows for offside. Jack Butland takes the indirect free kick.
Jared Sims is penalised for a handball. Jack Butland takes the free kick.
Jamie Griffiths concedes a free kick for a foul on Marlon Pack. Luke Garbutt takes the direct free kick.
A cross is delivered by Sido Jombati, save by Jake Cole.
Conor Hourihane takes a shot. Save by Jack Butland.
Jared Sims gives away a free kick for an unfair challenge on Joshua Low. Sido Jombati takes the direct free kick.
Conor Hourihane provided the assist for the goal.
25:47 Goal scored
Goal - Jared Sims - Cheltenham 0 - 1 Plymouth Jared Sims grabs a headed goal from inside the penalty area. Cheltenham 0-1 Plymouth.
A cross is delivered by Simon Walton, save by Jack Butland. Luke Young decides to take a short corner.
The ball is swung over by Joshua Low, Darryl Duffy takes a shot.
Booking The referee shows Carl Fletcher a yellow card for unsporting behaviour.
Free kick awarded for a foul by Luke Summerfield on Carl Fletcher. Direct free kick taken by Onismor Bhasera.
Corner taken right-footed by Marlon Pack, free kick awarded for a foul by Alan Bennett on Ben Gibson. Direct free kick taken by Jake Cole.
Sido Jombati crosses the ball.
Shot by Luke Summerfield from outside the box goes over the target.
Darryl Duffy fouled by Curtis Nelson, the ref awards a free kick. Luke Garbutt restarts play with the free kick.
Conor Hourihane produces a cross, Header from close range by Sido Jombati misses to the right of the goal.
Headed cross/shot by Luke Garbutt,
Jamie Griffiths is flagged offside by the assistant referee. Jack Butland takes the free kick.
Darryl Duffy concedes a free kick for a foul on Simon Walton. Jake Cole takes the direct free kick.
Carl Fletcher gives away a free kick for an unfair challenge on Jeff Goulding. Free kick taken by Marlon Pack.
The referee blows for offside against Darryl Duffy. Jake Cole takes the indirect free kick.
Corner taken left-footed by Luke Garbutt, clearance by Luke Young.
Shot by Conor Hourihane. Save by Jack Butland.
Kaid Mohamed produces a left-footed shot from just outside the box that misses to the right of the target.
Carl Fletcher takes a shot.
The assistant referee signals for offside against Jamie Griffiths. Jack Butland restarts play with the free kick.
Free kick awarded for an unfair challenge on Durrell Berry by Steve Elliott. Direct free kick taken by Jake Cole.
The game kicks-off.
Live text and data provided by The Press Association
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.