Gavin Tomlin scored an injury-time winner as Gillingham beat Hereford in a remarkable game.
United led 2-0 with a Rob Purdie penalty and Tom Barkhuizen close-ranger but Chris Whelpdale's strike and Danny Kedwell's header levelled it.
Barkhuizen's shot and Will Evans's header again put the Bulls in control but Charlie Lee's brace in the last 10 minutes levelled the scores.
Purdie had an injury-time penalty saved before Tomlin volleyed home.
Relegation-battlers Hereford had looked good for a surprise win when they led by two within seven minutes.
Barkhuizen was fouled by Jack Evans for Purdie's penalty before the Blackpool loanee scored himself.
Whelpdale and Kedwell netted just either side of the break to level the scores for the promotion-chasing hosts.
But Hereford rallied and Barkhuizen bundled home before Evans's 80th-minute goal.
Substitute Lee scored within 60 seconds and again with two minutes remaining to send the teams level going into injury-time.
Hereford had a golden chance to win it when Paulo Gazzaniga fouled Barkhuizen but the goalkeeper made amends with a save from Purdie's penalty before Tomlin's volley sealed one of the most dramatic wins of the season.
VIEWS FROM THE DRESSING ROOM
Gillingham manager Andy Hessenthaler told BBC Radio Kent:
"I don't really want to go through that again. I'm thankful for the win but there's lots of lessons to be learned tonight.
"The boys kept believing and a lot of teams would have crumbled but in the end we got a win which was unbelievable.
"You're confident in Gavin Tomlin because he's a goalscorer and he's an intelligent player. My reaction? I've never jumped so high.
"It was entertainment but for a manager it was exhausting at times."
Hereford manager Jamie Pitman told BBC Hereford and Worcester:
"The lads worked hard but you can't give goals away like we have.
"It was a sending off on Sam Clukas, it's a disgraceful tackle, how the referee cannot warrant it was a goalscoring opportunity is beyond me.
"Things like that cost people their jobs, my boys are in there and they have families and kids to look after, so have I and that could cost me my job."
Live Text Commentary
Full Time The match has reached full-time.90:00 +3:58 Booking
Booking Booking for Danny Kedwell for unsporting behaviour.90:00 +2:56
Assist by Lewis Montrose.90:00 +2:56 Goal scored
Goal! - Gavin Tomlin - Gillingham 5 - 4 Hereford Gavin Tomlin gets on the score sheet with a goal from deep inside the penalty box high into the middle of the goal. Gillingham 5-4 Hereford.90:00 +1:24
Direct effort from the free kick comes in from Kenny Lunt.90:00 +1:24 Booking
Booking Caution for Lewis Montrose.90:00 +1:19
Lewis Montrose concedes a free kick for a foul on Will Evans.90:00 +0:32
Inswinging corner taken by Tom Barkhuizen from the left by-line played to the near post, Paulo Gazzaniga makes a save.90:00 +0:19 Penalty
Power penalty taken by Robert Purdie saved by Paulo Gazzaniga. Gillingham 4-4 Hereford.89:33 Booking
Booking Paulo Gazzaniga goes into the book for unsporting behaviour.89:05
Paulo Gazzaniga challenges Tom Barkhuizen unfairly and gives away a Penalty.87:16
Assist by Oliver Lee.87:16 Goal scored
Goal! - Charlie Lee - Gillingham 4 - 4 Hereford Charlie Lee gets on the score sheet with a goal from inside the box high into the middle of the goal. Gillingham 4-4 Hereford.86:08
Short corner taken by Tom Barkhuizen from the left by-line.85:52
Shot from 25 yards by Will Evans. Paulo Gazzaniga makes a brilliant save.84:42
Free kick awarded for an unfair challenge on Danny Jackman by Joe Colbeck. Paulo Gazzaniga takes the direct free kick.83:30
Corner taken by Tom Barkhuizen, Paulo Gazzaniga makes a save.82:57
Outswinging corner taken by Tom Barkhuizen from the right by-line.81:32
Indirect free kick taken by David Cornell.81:32 Substitution
Substitution Joe Colbeck replaces Lyle Taylor.
The assistant referee signals for offside against Lewis Montrose.80:06 Goal scored
Goal! - Charlie Lee - Gillingham 3 - 4 Hereford Charlie Lee grabs a goal from just outside the penalty area to the bottom right corner of the goal. Gillingham 3-4 Hereford.79:38
The ball is crossed by Byron Anthony,79:14
Byron Anthony provided the assist for the goal.79:14 Goal scored
Goal! - Will Evans - Gillingham 2 - 4 Hereford Will Evans finds the net with a headed goal from inside the penalty area. Gillingham 2-4 Hereford.77:25
David Cornell restarts play with the free kick.77:25 Booking
Booking Charlie Lee is cautioned.77:13
Samuel Clucas fouled by Charlie Lee, the ref awards a free kick.76:11 Substitution
Substitution Lewis Montrose comes on in place of Curtis Weston.76:11 Substitution
Substitution Chris Whelpdale goes off and Oliver Lee comes on.75:36
Tom Barkhuizen concedes a free kick for a foul on Danny Jackman. Paulo Gazzaniga takes the direct free kick.74:04
Charlie Lee has a headed effort at goal from close in which goes wide of the left-hand upright.73:45
Corner taken by Gavin Tomlin from the left by-line.73:30 Substitution
Substitution Samuel Clucas on for Daniel Williams.70:40
Free kick awarded for a foul by Michael Townsend on Jo Kuffour. Direct free kick taken by Curtis Weston.69:19
Assist by Will Evans.69:19 Goal scored
Goal! - Tom Barkhuizen - Gillingham 2 - 3 Hereford Tom Barkhuizen finds the back of the net with a goal from inside the box low into the middle of the goal. Gillingham 2-3 Hereford.69:01
Will Evans takes the chance to get an effort at goal. Save by Paulo Gazzaniga.67:48
Will Evans concedes a free kick for a foul on Gavin Tomlin. Free kick crossed by Danny Jackman, Simon Clist manages to make a clearance.66:59
Shot by Will Evans from outside the box goes over the net.63:35
The official flags Tom Barkhuizen offside. Paulo Gazzaniga takes the indirect free kick.62:47
Effort from the edge of the penalty box by Gavin Tomlin goes wide of the right-hand upright.62:08
Delroy Facey challenges Connor Essam unfairly and gives away a free kick. Direct free kick taken by Paulo Gazzaniga.60:22
Jack Payne fouled by Will Evans, the ref awards a free kick. Curtis Weston takes the free kick.59:56
Curtis Weston takes a shot from 20 yards. Save by David Cornell.59:33
Drilled right-footed shot by Tom Barkhuizen. Save made by Paulo Gazzaniga.58:08
Effort from a long way out by Delroy Facey goes wide left of the net.57:35
Long range shot from Jack Payne misses to the left of the target.55:38
Effort on goal by Tom Barkhuizen from 6 yards. Comfortable save by Paulo Gazzaniga.51:23
Foul by Will Evans on Simon King, free kick awarded. Paulo Gazzaniga restarts play with the free kick.49:35
Danny Kedwell fouled by Byron Anthony, the ref awards a free kick. Danny Jackman restarts play with the free kick.48:59
Lyle Taylor has a shot on goal from outside the box which misses left.48:27
Foul by Michael Townsend on Danny Kedwell, free kick awarded. Free kick taken by Paulo Gazzaniga.47:17
Will Evans gives away a free kick for an unfair challenge on Simon King. Paulo Gazzaniga takes the direct free kick.45:33
Charlie Lee provided the assist for the goal.45:33 Goal scored
Goal! - Danny Kedwell - Gillingham 2 - 2 Hereford Danny Kedwell scores a headed goal from close in. Gillingham 2-2 Hereford.45:01
The match restarts for the second half.45:01 Substitution
Substitution Jack Evans goes off and Charlie Lee comes on.45:00 +3:40 Half time
Half Time The players leave the pitch at half time.45:00 +2:25
Chris Whelpdale has a header from close in and clears the crossbar.45:00 +1:48
Michael Townsend challenges Jo Kuffour unfairly and gives away a free kick. Danny Jackman takes the direct free kick.45:00 +1:28
Jo Kuffour has an effort at goal from just outside the penalty box which goes wide of the left-hand post.45:00 +0:12 Booking
Booking David Cornell is cautioned.44:10
Assist on the goal came from Danny Jackman.44:10 Goal scored
Goal! - Chris Whelpdale - Gillingham 1 - 2 Hereford Chris Whelpdale grabs a goal from inside the box high into the middle of the goal. Gillingham 1-2 Hereford.44:06
Danny Jackman crosses the ball,43:09
Chris Whelpdale has a shot on goal from outside the area which misses right.42:33
Corner taken by Danny Jackman, clearance by Delroy Facey.38:34
Lyle Taylor takes a shot. Save by Paulo Gazzaniga.37:59
Shot by Will Evans from 25 yards. Blocked by Simon King.36:21
Gavin Tomlin has an effort at goal from outside the penalty area which goes wide right of the goal.34:48
Robert Purdie concedes a free kick for a foul on Danny Jackman. Free kick taken by Jack Payne.31:57
The official flags Gavin Tomlin offside. David Cornell takes the indirect free kick.30:39
The referee gives a free kick against Tom Barkhuizen for handball. Direct free kick taken by Paulo Gazzaniga.29:55
Danny Kedwell takes a shot. David Cornell makes a save.26:48
Foul by Simon Clist on Jack Evans, free kick awarded. Direct free kick taken by Chris Whelpdale.25:05
Inswinging corner taken right-footed by Gavin Tomlin played to the near post, Robert Purdie manages to make a clearance.24:53
Gavin Tomlin has an effort at goal from 12 yards. David Cornell makes a save.22:53
Foul by Danny Jackman on Tom Barkhuizen, free kick awarded. The free kick is delivered right-footed by Tom Barkhuizen from right wing, Danny Kedwell manages to make a clearance.21:35
Corner taken by Tom Barkhuizen from the right by-line.21:16
Will Evans fouled by Jack Evans, the ref awards a free kick. Tom Barkhuizen crosses the ball from the free kick right-footed from left by-line, Simon King makes a clearance.19:30
Shot by Chris Whelpdale from outside the box goes over the net.12:59
Jo Kuffour produces a drilled left-footed shot from deep inside the area which goes wide of the left-hand post.11:11
Free kick awarded for a foul by Delroy Facey on Connor Essam. Simon King restarts play with the free kick.10:30
Will Evans takes a long range shot clearing the bar.7:00 Goal scored
Goal! - Tom Barkhuizen - Gillingham 0 - 2 Hereford Tom Barkhuizen scores a goal from close range to the bottom right corner of the goal. Gillingham 0-2 Hereford.3:07
Assist on the goal came from Tom Barkhuizen.3:07 Goal scored
Goal! - Robert Purdie - Gillingham 0 - 1 Hereford Placed penalty scored by Robert Purdie. Gillingham 0-1 Hereford.2:14
Jack Evans challenges Tom Barkhuizen unfairly and gives away a Penalty.0:00
The ref blows the whistle to begin the match.
Live data and text provided by our data suppliers