Christian Benteke scored from the penalty spot in injury time to give Tim Sherwood his first win as Aston Villa manager and lift them out of the Premier League relegation zone.
Saido Berahino's 18th goal of the season looked to have extended Villa's run without victory to 13 league games after he equalised Gabriel Agbonlahor's first-half goal.
But with only seconds remaining, West Brom keeper Ben Foster rushed out to bring down Matthew Lowton.
Benteke kept his cool to slide in the spot-kick to give Villa a 2-1 win and three points for the first time since 7 December.
Sherwood, who lost his first two games at home to Stoke City and at Newcastle United, added: "We have improved every game and that was our best performance.
"It was a magnificent feeling. It was a sickener when we lost in the last minute against Stoke and it's a much nicer feeling to be on the other side."
Hull 1-1 Sunderland
Sunderland manager Gus Poyet was sent to the stands as his team came from behind to earn a draw at Hull City.
The result leaves both sides still separated by a point, with Hull 15th and Sunderland 16th in the Premier League table.
City boss Steve Bruce and Poyet had to be separated after a touchline argument as the Sunderland boss left the pitch after kicking a bucket full of water bottles.
Southampton 1-0 Crystal Palace
Sadio Mane ended Southampton's long run withour scoring at home by netting a late winner against Crystal Palace to send the Saints fifth in the Premier League.
The Senegalese chipped over Palace keeper Julian Speroni after he could not hold James Ward-Prowse's shot.