More uniformed security officers may need to be deployed at tracks after outbreaks of violence, say racecourse bosses.
Fighting broke out at Ascot on Saturday, a week after police said around 50 people were involved in "multiple altercations" at Goodwood.
Racecourse Association (RCA) chief executive Stephen Atkin said tracks would need to assess security measures.
"It means employing the right level of security staff," he told BBC 5 live.
"The question you have to consider is how many of those should be in high visibility and how many in plain clothes. That is something racecourses may want to look at and step up the level of visibility."
The British Horseracing Authority (BHA) says troublemakers could be banned from racecourses.
"We have been in contact with the RCA this weekend and we know that courses will take into account the incidents as they make their security plans for each meeting," it said.
"The BHA does have the power to exclude individuals from courses and will use it where it can."
At Ascot, a window was smashed at the course as trouble broke out in a stand and footage was later widely circulated on social media of racegoers trading blows outside the entrance.
Police said two men with minor injuries were treated at the scene and no arrests had been made.
Racecourse officials said security had been increased and the incident was dealt with quickly. They have pledged to put in place "appropriate measures" for Royal Ascot next month, although did not expand on details.
Atkin acknowledged that drugs, as well as alcohol, were an issue at some meetings.
"There is very considerable training which the RCA helps run of racecourse staff about serving alcohol in bars and making drinking water freely available and advising people how to pace themselves," he said.
"With the issue of drugs, there are regular checks on areas that may be prone to drug misuse. The RCA and our racecourse experience group are in regular touch with the police officers' association about how we can best tackle that. "
The BHA indicated after the Goodwood incident that "crowd control and behaviour" could be taken into consideration when a racecourse licence is issued under new plans.
York racecourse believes it has measures in place for this week's Dante meeting to prevent trouble, including the use of sniffer dogs.
"We're obviously aware of the unsavoury incidents and we will take all the sensible precautions to take racegoers' safety where it should be," said James Brennan, York's head of sponsorship and marketing.
Craig Newman, who filmed the Ascot brawl:
There were 20 or 30 people throwing punches. It was after the races had finished.
You could see moving through the crowd how many people were off their faces, whether it was drink or drugs. You could tell it was going to happen.
There was supposedly plain-clothed security, but there was no visible security I could see. Maybe that is what they need, people with fluorescent jackets.
BBC horse racing correspondent Cornelius Lysaght
There have, of course, been incidents in the past but not on this kind of scale, with this kind of widespread publicity - the number looking at the frightening images of the Ascot fight online is well into seven figures.
It's as much the talk of the racing community as who'll win next month's Derby at the moment, especially as the two tracks concerned are Ascot and Goodwood, both of which trade on their 'upmarket' reputations.
The word that is used most frequently when discussing this is cocaine. People are talking not so much about drink and other things causing this type of problem, but drugs being the number one factor.
One of the priorities for racecourses this summer - and this is happening - is to bring in sniffer dogs to catch people with drugs; I was struck during a visit to the gents at a recent fixture that there was a queue for the privacy of cubicles when urinals were free, a tell-tale sign I'm told.
More overt security - police officers, with the powers of arrest and the consequences that can follow - is on the cards, and needed, to deter people who wouldn't take any notice of a well-meaning security person in a bowler hat at Ascot or Panama at Goodwood or even a hi-vis jacket.
With Royal Ascot and Glorious Goodwood not far off, and events like the Derby on the horizon, racing officials must have this at the very top of the agenda, to stamp it out quickly before the sport gets a reputation as the go-to place for this type of activity, putting off the law-abiding majority.
Meanwhile, those identified as the culprits should be hit hard and swiftly with bans from all racecourses for life.