"That's three times now I've been clipped and booked for no reason," said the Spurs winger, 23.
"People keep saying I'm diving, but if there's contact it's not diving. Referees need to look more closely."
Sunderland opened the scoring shortly before half-time through a close-range strike from defender John O'Shea.
But Tottenham equalised through Carlos Cuellar's own goal before Aaron Lennon fired the winner moments later.
The victory moves Spurs up to third in the Premier League table, but they will be without Bale for the visit of Reading.
"People think I'm diving when there's contact," said the Welshman, who fell when challenged in the penalty area by Sunderland's Craig Gardner after 80 minutes at the Stadium of Light.
"If there's contact, it's a penalty or free-kick. There's nothing I can do. What do you want me to do? Hit my head on the floor? I have to put my hands down to protect myself. If people kick me, I'll go down.
Bale booking 'tremendously unfair'
"People keep saying I'm diving but it doesn't bother me whatsoever. It's a foul and I know it myself."
Bale was also yellow-carded twice last season for diving.
Earlier in the match, Bale's team-mate Jermain Defoe escaped punishment when he appeared to go down without being touched.
Tottenham manager Andre Villas-Boas felt the two incidents show referees may be singling out Bale for special treatment.
"It looks like persecution," Villas-Boas said. "It was a big mistake from the referee."
Yellow cards accrued in the first half of the season are after wiped after 31 December but Bale's fifth comes before the cut-off.
"If you see the images, Gardner is not even focusing on the ball, he goes in and clips him," Villas-Boas added.
"It doesn't mar the win but Gareth was one game from being clean from yellow cards and it's extremely unfair that he misses Reading."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.