Prime Minister Boris Johnson has said he does not think the song Swing Low, Sweet Chariot should be banned after the Rugby Football Union announced it would review its use.
The RFU said many people were not aware of the song's links with slavery.
But the PM said people should focus less on symbols and more on the substance of racism.
Mr Johnson said he did not think there should be "any sort of prohibition" on singing the song.
He added: "Frankly I think what people need to do is focus less on the symbols of discrimination... all these issues that people are now raising to do with statues and songs and so on - I can see why they're very emotive, I understand that.
"But what I want to focus on is the substance of the issue."
He added that he "certainly didn't think there should be any sort of prohibition on singing [Swing Low, Sweet Chariot]".
"Nobody, as far as I'm concerned, seems to know the words," Mr Johnson said.
"Before we start complaining about Swing Low, Sweet Chariot I'd like to know what the rest of the words are."
Former rugby league and union wing Martin Offiah, who was playing during the song's first known use at Twickenham in 1987, has welcomed the RFU review but does not want it banned either.
Offiah told Radio 5 live: "The song is not really what the issue is here - the issue is about diversity and inclusion.
"I think this is the first step as we progress towards change."
The song is believed to have been sung at rugby clubs since the 1960s but came to prominence at Twickenham in 1987, when Offiah played in the Middlesex Sevens tournament.
It is thought Swing Low, Sweet Chariot was written in the mid-19th century by Wallace Willis, who was a black slave.
"It's definitely an emotional piece of music, very emotive, it stirs up feelings and that's probably something to do with its history," Offiah said.
"That history is probably not that well known by a lot of people in the UK. I champion the RFU reviewing it, I wouldn't support the banning of such a song. When you do try to ban things like that it just makes the song more divisive.
"If this review leads to the RFU putting a positive spin on this song, engaging with ethnic communities, looking at the rooms where decisions are made in the RFU and addressing those issues, that's what we actually want."
Former England captain Maxine Edwards believes the RFU has bigger issues to face than fans' use of the song.
Edwards said: "I think it is interesting that the RFU has decided to review this song and have discussions about its appropriateness, as part of their bigger process of reviewing their approach to the representation of people of BAME backgrounds within their organisational structure at all levels and taking part in their sport.
"I would, however, ask why this is the first thing that they have on their list to review as part of this review process?
"It is complicated, but it is really by no means the biggest issue that the RFU needs to address."
Last week World Cup winner Maggie Alphonsi - the only black person on the RFU council - said that the death of George Floyd in the United States had led to "powerful conversations".
RFU chief executive Bill Sweeney has promised to increase diversity in the organisation, saying: "We have undertaken some very good initiatives at the grassroots level to encourage more diverse participation. However, that in itself is not enough.
"We need to do more to achieve diversity across all areas of the game, including administration."
- Football Daily: When will the Champions League restart?
- Grounded with Louis Theroux: Louis sits down with Troy Deeney