Horstmann went over for Worcester's crucial second try
Worcester edged out Bedford to book a place in the Championship play-off final in a dramatic climax at Sixways.
Four James Pritchard penalties and an Edd Thrower try put Bedford 19-3 ahead after 25 minutes.
Andy Goode's touchdown then started Worcester's comeback as the home side's powerful pack gained the upper hand.
But the Warriors left it late, Kai Horstmann forcing over for the crucial try, coolly converted by replacement Joe Carlisle, with two minutes left.
The home side thought they had won it moments earlier when lock Graham Kitchener, already celebrating his try, stormed over the line only to drop the ball as he was tackled, trying to run round closer to the posts.
The despair on Kitchener's face was matched by those within the partisan Sixways crowd, aware that the dream of a swift return to the Premiership was rapidly fading.
But despite the error, Worcester managed to keep their nerve. And, with Bedford a man light following a yellow card to Sam Walsh, they dominated a series of five-metre scrums from where Horstmann powered over for their second vital try.
Even then, there was more drama when, from Myles Dorrian's last-ditch drop-goal attempt, Tom Bedford thought he had scored a winning try for the Blues.
But TV replays showed that Dorrian's drop goal attempt had bounced behind the dead ball line.
Having dominated the Championship this season with only one defeat in 22 league games, Worcester were clear favourites going into the semi-final clash.
But a string of errors allowed Bedford captain Pritchard three easy penalties.
Scrum-half Will Chudley was then the catalyst for Bedford's opening try as the visitors stunned Sixways by going 16-0 up inside 18 minutes.
“ I don't think we could have argued if Bedford had won”
Richard HillWorcester coach
Chudley found space at a line-out to break through Worcester's defence, before the ball found its way to Thrower, who went over in the corner, Pritchard adding the extras.
Worcester finally got on the scoresheet on 20 minutes when Goode took advantage of a Bedford infringement to slot over a penalty.
Pritchard immediately responded with a penalty to make it 3-19.
But, on 34 minutes, Goode was on hand to finish off a fine break by full-back Tom Arscott by crossing for Worcester's first try, the former England international himself adding the conversion.
And the deficit was reduced to just six points by half-time after Goode slotted a second peanlty to make it 13-19.
Pritchard struck a fifth penalty shortly after the restart, increasing the level of tension around Sixways as the Blues moved nine points clear.
But, with 20 minutes left and another Goode penalty on the scoreboard, Richard Hill's men really started to throw everything at their opponents.
And, after what looked like heartbreak when replacement Kitchener spilled the ball five minutes from time, the Warriors were finally rewarded.
Bedford's valiant defence was at last broken when captain Horstmann dived over from close range.
The tension was heightened by an ugly outburst of fistitcuffs which saw both Kitchener and rellow replacement Sean Tomes yellow carded before Carlisle's left boot nervelessly slotted the winning kick.
Having then withstood that late drop-goal scare at the other end, Worcester booked a Championship play-off final meeting with Cornish Pirates,
who beat London Welsh 18-10.
They meet over two legs, the first of them to be held in Penzance on Wednesday 11 May at the Mennaye Field (1945 BST).
Warriors head coach Richard Hill told BBC Hereford & Worcester:
"I don't think we could have argued if Bedford had won. We had enough possession in the second half but we didn't establish control and weren't as dominant up front as we needed to be.
"We started off poorly, we were too anxious and piling into the breakdown and committing penalties.
"What we saw was a lot of pressure on the Warriors and we made those errors. We know we have escaped and I would hope we won't play as badly as that in the final."
Bedford director of rugby Mike Rayer:
"We brought a massive amount of attitude and defence to the game and no little attack but sadly we didn't get the result.
"We played some great rugby in the first half but, in the second half, we didn't have enough field position to keep the scoreboard ticking over."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.