A car owned by a family with a second home was daubed with "Go Home" amid a series of "vigilante" attacks.
The attack at Bigbury-on-Sea, in Devon, was one of a range of similar incidents across the region, said police.
Victim Tony Willis, who arrived with his family before the coronavirus travel ban, said he was sick of being targeted.
Devon and Cornwall Chief Constable Shaun Sawyer said the behaviour was "unacceptable".
"Over the past week we have had Go Home daubed in the dirt on the back windscreen of our car, leaflets telling us to do the same and someone haranguing me to 'Go Home' from the street outside our house," said semi-retired businessman Mr Willis.
"This is pure harassment and in another context would be considered a hate crime."
The family bought their house 10 years ago and came down for a birthday celebration and to be near an elderly relative.
"Since the lockdown we decided to stay here because it would be non-essential travel to go back," he said.
"I spend a lot of time in Devon, I carry on a business down here and my wife's from Devon.
"We are following all the rules, we are self-isolating and we only go out when we have to for provisions.
"We love it down here and we're not going to be run out of town by a few idiots."
Mr Sawyer said the behaviour towards the Willis family was "shocking, horrible and unacceptable".
"This is a family who are probably just as frightened and concerned as everyone else," he said.
"If they are already here we should welcome them and make them feel part of our community."
He said there had been "some horrible incidents of hate crime".
"But this is not a time to turn person against person and for us in the South West to turn against the people of London or vice versa," Mr Sawyer warned.
Beth Huntley, Bigbury Parish Council chair, said 63% of properties in the village were holiday-lets or second homes and locals were concerned about the impact of coronavirus on the village's elderly population.
"We hope the community is free of the virus, but every newcomer brings a threat to the older community," she said.
"It would be lovely if people who own holiday homes stayed close to their homes.
"But I think a conciliatory approach is always a good idea."