Residents have launched a campaign to save a centuries-old building that has stood empty for about 15 years.
Island House, in Laugharne, Carmarthenshire, dates to the Tudor period.
It became empty when the owner died and it has since been put on an at-risk register.
Denize McIntyre, from Save Island House Campaign, said people would not let the Grade II*-listed building "fall down".
Councillor Phillip Hughes said the county council was also trying to safeguard its future.
BBC Wales has been unable to contact the executor of the estate of the last occupant to seek a comment.
"We love this place," said Ms McIntyre.
"It's not just a tourist destination, it's a vibrant, living community and we don't want it being taken for granted - leaving this building to just crumble and fall down."
The Society for the Protection of Ancient Buildings said it was "a very sorry state of affairs" as the building has been on an at-risk register since 2011.
Mr Hughes, Carmarthenshire council executive board member for enforcement, said "complex discussion are on-going as the council looks to safeguard the long-term future of the building".
"Although the council has resolved to potentially obtain the property through a compulsory purchase order, we continue to explore all available options to secure the necessary improvement works," he said.