Harlequins produced a resolute display to see off indisciplined Gloucester.
Billy Twelvetrees' try at The Stoop earned the visitors an early lead, but Danny Care and Nick Easter went over for Quins after Jimmy Cowan was sin-binned for a 20-5 half-time advantage.
Luke Wallace's try then put Quins 22 points ahead, only for Jonny May to go over despite Gloucester being down to 13 men after two more yellow cards.
Twelvetrees added a late second try, but Quins held firm to go third.
It has been a real turnaround in fortunes of late for Harlequins, who had lost three of their opening five Premiership games.
Happy anniversary at The Stoop
Harlequins celebrated the 50th anniversary of the first Quins match to be played at The Stoop
Evans kicked them into an early lead after Gloucester were penalised for offside, but their advantage was soon wiped out when Twelvetrees went over after Huia Edmonds broke away from the maul and threw a clever dummy to create the opportunity.
But Evans fired another penalty over the posts to put Quins back in front and, with the hosts on top, only an illegal intervention by Jimmy Cowan - for which he was sin-binned - prevented Care going over.
Fly-half Evans this time turned down the penalty in favour of kicking for touch and Quins were immediately rewarded for their ambition as Easter forced his way over following the line-out for a converted try.
With Cowan still to return to the field, the home side touched down again as Care burst through between the posts under pressure to score a deserved try - Evans again adding the extras.
Harlequins looked to be out of sight just four minutes into their second period, with further Gloucester indiscipline proving costly.
Tom Savage was sent to the sin bin for taking George Robson out in mid-air, and Luke Wallace touched down at the base of a rolling maul in the aftermath, with Evans converting once more.
Replacement scrum-half Martyn Thomas was soon to become the third Gloucester player to see yellow but, despite being two men light, Jonny May pounced on a loose ball and raced clear to give the visitors hope, with Freddie Burns converting.
Harlequins then had George Robson sin-binned for being offside at a ruck, but they looked to be coasting to victory when Tom Williams touched down, but the TV match official ruled it out for a forward pass.
Twelvetrees latched on to Burns' chip forward to set up an interesting final five minutes and May almost went over again in a frantic finale to the match, but Evans scrambled back to cover the winger's kick forward and prevent the visitors earning a bonus point.
VIEW FROM THE DRESSING ROOM
Harlequins director of rugby Conor O'Shea:
"We let the momentum go in the game, when we were in total control. I don't understand why we took our foot off the pedal.
"That is three consecutive wins in the Premiership and we are back in the top four.
"We are digging our way back into the Premiership but we are going to have to be more ruthless."
Gloucester director of rugby Nigel Davies told BBC Radio Gloucestershire:
"We're just not accurate enough at the moment and because of that lack of accuracy we're putting pressure on ourselves and that's causing discipline issues.
"When you're playing sides like Harlequins and you go down to 13 or 14 men you're going to concede points and it's not good enough.
"The players have got to look at themselves and we've got to look at what we're doing. We must make sure we aren't conceding these penalties like we are at the moment because it's absolutely killing us.
"There's a lot of hunger and a lot of desire and, again, we showed a lot of character which this group has got but it's really not enough and the boys know that."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.