Haringey Borough's chairman says it is "soul-destroying" that their FA Cup tie against Yeovil Town was abandoned.
Saturday's game was called off after Haringey's manager took his team off the field amid reports of racial abuse from Yeovil fans.
"I am of the view that we had no choice. We could not carry on and play football yesterday," Aki Achillea told PA Sport.
The Metropolitan Police say they are investigating the matter.
Officers are also investigating after items were reportedly thrown on to the playing area, a Metropolitan Police spokesperson told BBC Sport.
There have been no arrests yet and inquiries are continuing.
- Bristol City investigate claims of racism by away fans at Luton
- Hearts investigate claims of racism towards Rangers striker
The match at Haringey's Coles Park Stadium was in the fourth qualifying round of the FA Cup, with the winner set to progress to the first round proper.
Haringey goalkeeper Valery Douglas Pajetat was reportedly spat at and hit by an object thrown from the Yeovil Town end.
Defender Coby Rowe was then "racially abused", according to Haringey boss Tom Loizou, who said "there was no way I could let him continue".
Avon and Somerset Police will be assisting the Met, and have asked anyone with information or video footage to contact them.
'There were people with tears in their eyes'
Achillea, chairman of the Isthmian League club, cannot understand what his players were subjected to on Saturday.
"It is soul-destroying because of the effort we've put in to encourage so many people from our area, from so many cultures, to come and watch what we have on offer," he added.
"If you look at the make-up of our team, we have six or seven Greek-Cypriots, I'm Greek-Cypriot, the manager is.
"We have a multitude of West Indians, Africans, Portuguese. We have virtually every nationality as part of the playing squad, and to see them have to be subjected to that, it is soul-destroying.
"Our players' heads had gone, there were people with tears in their eyes in the dressing room. These are young kids and they shouldn't be subjected to that.
"Ultimately the impact upon them was such that they couldn't carry on and play football, because it's gone beyond football.
"I'm really, really disappointed for the way it worked out for all of us. To have it wrecked by what was definitely a very small minority of people who attach themselves to the club and call themselves fans.
"But they're not the real Yeovil fans, we know that.
"A lot of them spoke to us immediately after the game and sympathised with us, and effectively congratulated us on taking the stand we did.
"Their management, their chairman, their players, they were a credit to the club - it's this small minority that have tarnished the name of Yeovil."
The Football Association is also investigating and said it was "deeply concerned" about the allegations.
"There is no room for discrimination in our game and we are working with the match officials and the relevant authorities, as a matter of urgency, to fully establish the facts and take the appropriate steps," said an FA statement.
The incident came four days after England's Euro 2020 qualifier in Bulgaria was halted twice as fans were warned about racist behaviour, including Nazi salutes and monkey chanting.
"It's very distressing," Loizou told BBC Radio 5 Live. "The abuse a few of my players got was disgusting.
"Yeovil's players and manager were different class. Their team tried to calm their supporters down, they tried their best and they supported us - they said 'if you're walking off, we're walking off with you'.
"I took the decision to take my team off and I don't want Yeovil Town to get punished for it. If we get thrown out of the FA Cup and they go through, there is no hard feelings there.
"I have not done it for any other reason than looking into my players' faces and seeing how distraught they were. They are not used to this."
What happened at Haringey v Yeovil?
Visitors Yeovil were leading 1-0 through a Rhys Murphy penalty when the game was halted in the 64th minute.
There was a long delay for that spot-kick to be taken, with Haringey keeper Pajetat reportedly struck by an object from the stands.
Shortly after Murphy scored, play was suspended as the hosts left the field. About 35 minutes later, it was confirmed the match had officially been abandoned.
Haringey said on Twitter: "Game has been abandoned following racial abuse. Horrendous afternoon.
"It must be said that 99.9% of [Yeovil] fans are also disgusted by what's happened as much as we are. One club, one community."
In a statement Yeovil said the club "will not accept racism or discrimination in any form" and that they will "be cooperating with the authorities and our friends at Haringey".
'The boys were disgusted'
Former Haringey left-back Michael O'Donoghue was at the game and went into the dressing room with the team when they came off.
The 23-year-old, who played for Borough for three and a half years before leaving last season to join Concord Rangers, told the PA news agency: "As soon as they scored it was starting to kick off again and you could see the Haringey players were getting abused and felt unsettled.
"The referee was seen removing the bottles thrown at the players and then the managers of both teams went on and escorted their teams off the pitch.
"I went into the changing room with the boys, the referee called in both captains and managers to have a talk about what to do next. The boys made a group decision to make a stand and not go back out to play.
"The officials and Yeovil manager came into the Haringey changing room and said they will back whatever decision the Haringey boys made, so that was class from them.
"The boys were disgusted and you could see a few were distraught from getting abused like that."