Using a former military base to house asylum seekers is "completely inappropriate for vulnerable people who have fled terror and suffering", campaigners have said.
About 100 people who are against plans for them to be housed in the Penally Army centre near Tenby gathered there on Sunday.
The campaigners say they back housing asylum seekers in Pembrokeshire.
The Home Office said it was working to find suitable accommodation.
The Penally Army centre is one of several sites being considered for housing asylum seekers,.
The UK's Welsh Secretary Simon Hart, also the local MP, has said he is seeking "further clarification".
Sunday's peaceful event was organised by Stand Up to Racism West Wales who say they "warmly welcome refugees to west Wales" but argue a former military facility is a "completely inappropriate" place to house them.
"We believe that refugees should be treated with the dignity that they deserve and offered decent housing, education and jobs," the group said in a statement.
"Forcing people into military bases and camps to live a 'living hell' is degrading, dangerous and potentially disastrous."
Nimi Trivedi said she joined the protest as she wanted to send a message of unity and hope.
"We're here to make a stand that the refugees and asylum seekers who are coming here are welcome," said Ms Trivedi, from Swansea.
The base was used by the army and Ministry of Defence until the Home office took it over.
Confirmation about the possible plans led to a protest from local residents on Tuesday.
The Police and Crime Commissioner for Dyfed-Powys Dafydd Llywelyn said he had written to Home Secretary Priti Patel over the issue.
The Welsh Government said in a statement: "The UK government didn't consult the local community, local services or us about their plans to use a military base in Wales for asylum seekers.
"We believe people fleeing war and conflict deserve the best possible start in a new country. We must do better."
The Home Office said it was working with the council and police to ensure "vulnerable asylum seekers" had suitable accommodation as facilities in south-east England were under strain.
"During these unprecedented times, the government is working with a range of partners and across departments to secure further accommodation and the MoD has offered use of some of its sites," said a Home Office official.
"When using contingency accommodation we work closely with organisations, including local authorities and law enforcement, throughout the process to ensure value for money and that vulnerable asylum seekers, who would otherwise be destitute, have suitable accommodation while their claims are processed."