Hundreds of families have been served eviction notices by the Ministry of Defence, according to a letter seen by BBC Wales.
Residents in an estate in Pembrokeshire are among those affected, and say the notices have caused them "sleepless nights".
Conservative Preseli Pembrokeshire MP Stephen Crabb said it was an appalling decision.
The MoD said tenants had been given 12 months' notice to leave.
It said it had a duty to dispose of properties not required for future service personnel. But the owner of the land says the MoD could dispose of 11,000 empty properties instead.
Families renting homes on the Cashfield Estate in Haverfordwest from the MoD are among those who have been served eviction notices - saying they need to leave by March 2021.
The homes were leased to civilian tenants by the MoD from a private company, Annington Homes Ltd.
Tenants were told the reason for the eviction notice was that the homes were being released back from the MoD to Annington Homes.
The MoD later issued a message to residents saying that "in cases of hardship" they could extend tenancies until September 2021 "on a case-by-case basis".
Annington Homes said while it was the freeholder of the properties, the decision to terminate the tenancies was "the MoD's alone and we were not consulted on it beforehand".
A spokesman added: "We are engaging with the MoD to understand their background to these notices being issued and to support the tenants affected."
'What if we go back into lockdown?'
Aden Cotterill, who has lived on the Cashfield Estate for three years, said it was a "disgusting" decision.
"You're talking forty plus homes being uprooted," he said.
"For that to happen during these times, during a global pandemic, and during a recession when everybody's got no money anyway is just unbelievable."
Mr Cotterill said his biggest concern is trying to find a new home.
"My other half is disabled so getting out and about isn't easy for her. In this area there are not many houses up for rent at the moment.
"What if we did go back into lockdown? Then there's even less income coming in. Are they going to even allow you to move house?
"My neighbour is worried she might end up homeless.
"Some of my neighbours have lost jobs and are currently unemployed, some of my neighbours have been affected by coronavirus and moving is the last thing on their mind."
He said that when they first rented the property they were told they were leased by the MoD from Annington Homes and that "the only time we would be expected to move out would be if they wanted to move the Army back in here, but they said that was highly unlikely."
"In an ideal world we'd like to see Annington Homes allow the houses back with sitting tenants in them already - even if they put the rent up, anything like that would be better than kicking us out."
In a letter seen by BBC Wales, from Annington Homes to the Defence Minister Jeremy Quin, the company said "nearly 350 families" in the UK have been affected by these evictions.
The letter said there was no requirement for the MoD to return these properties to Annington Homes and they "cannot fathom" the rationale behind the MoD's decision.
It also added that there were 11,000 empty properties owned by Annington, and leased by the MoD, which they believe the MoD could return instead of the hundreds with tenants living in them.
Preseli Pembrokeshire MP Mr Crabb said he was "appalled to find out that the Ministry of Defence had issued these eviction notices to families in my constituency.
"In Pembrokeshire there is shortage of decent, affordable properties to rent anyway.
"But given we're in the middle of pandemic which is also seeing an increase in the property market, people moving into the area from outside, buying up available homes, it's the worst possible moment to be receiving an eviction notice."
He called on the MoD to withdraw the notices and for a defence minister to "come to Parliament and explain what is going on".
What has the MoD said?
A MoD spokesman said: "Civilian tenants who rent these homes sign-up to two months' notice.
"However, in recognition of the extraordinary circumstances being faced by tenants during the Covid-19 pandemic we have extended the Notice to Vacate period to 12 months.
"This provides up to a year for the MoD to work with Annington Homes to whom the houses are being returned.
"We would like to enable homes to be transferred to Annington with sitting tenants but this would require their support. We are committed to working with them to try and achieve this objective".