Online cartoon Pepe the Frog has been added to the Anti-Defamation League (ADL)'s database of hate symbols.
Other logos cited as offensive by the ADL include the Swastika and the "Blood Drop Cross" of the Ku Klux Klan (KKK).
The anti-bigotry group said "racists and haters" had "taken a popular internet meme and twisted it".
Pepe has recently been depicted as Adolf Hitler and a member of the white supremacist KKK.
Pepe made his debut in 2005 in artist Matt Furie's "Boy's Club" cartoons. Since then, pictures of the creature have spread through the online communities 4chan, 8chan, and Reddit, where users can post an image for others to comment on.
These are mostly used to express emotions or experiences, but some racist and anti-Semitic versions have spread virally on Facebook and Twitter.
"These anti-Semites have no shame," ADL's chief executive Jonathan A Greenblatt said. "They are abusing the image of a cartoon character, one that might at first seem appealing, to harass and spread hatred on social media."
'You can't stump the Trump'
The Alt-Right movement, a disparate group of right-wing social media users, has repeatedly shared Pepe re-workings on social media.
The movement has been described as an alternative to mainstream conservatism, but its proponents have repeatedly abused Jewish Americans and portrayed white people as oppressed.
The Alt-Right has claimed Mr Trump as its presidential candidate of choice. The mogul has not approved it, but did re-tweet a caricature depicting him as Pepe the Frog in October 2015, with the caption "You Can't Stump the Trump".
The frog made headlines in mid-September when Hillary Clinton said most of Mr Trump's supporters belonged in a "basket of deplorables".
The Republican's son Donald Trump Jr shared an altered version of the movie poster for "The Expendables", which showed Pepe with his father and other conservatives, labelled "The Deplorables."
'That cartoon frog is sinister'
Mrs Clinton's team duly released a post titled "Donald Trump, Pepe the frog, and white supremacists: an explainer" claiming, "that cartoon frog is more sinister than you might realize".
In May 2016, the Daily Beast quoted an anonymous white nationalist saying that a campaign was under way to "reclaim Pepe from the normies" - internet slang for mainstream users.
The ADL has clarified that most Pepe memes are not racist, and that innocent versions will not be subject to the hate symbol designation.
"The mere fact of posting a Pepe meme does not mean that someone is racist or white supremacist," the group said.