More than half of small firms employing workers from the EU are worried that Brexit will stop them recruiting sufficiently skilled staff.
A report from the Federation of Small Businesses (FSB) has found that only 21% of small firms currently employ staff from an EU country.
But of those, 59% are concerned they will not be able to recruit the right staff in the future.
The FSB said securing the right of EU staff to remain in the UK was vital.
Without that, said the FSB's chairman Mike Cherry, small firms feared they would lose skilled staff who would be difficult to replace.
"EU workers are a vital part of our economy, helping to plug chronic skills gaps across a wide range of sectors, and filling jobs in an already tight labour market," he said.
"From packers, to mechanics, to graphic designers, small employers need to be able to hire the right person, for the right job at the right time."
The right of EU and non-EU citizens to come and work in the UK, after Brexit, is still entirely unclear.
Jonathan Bates is the director of Photon Energy, a solar power company which derives more than a-fifth of its highly-skilled workforce from the EU.
"To have access to those people restricted would cause us significant difficulties," he told the BBC.
If the UK's existing visa scheme was applied to EU workers it would be "time consuming, cumbersome and expensive" for his business, Mr Bates said.
For its part, the FSB says the government should guarantee as soon as possible the right of existing EU staff to stay here.
And there should be a transition period of at least three years to whatever new immigration rules might affect fresh workers from the EU.
"There can't be a sudden cliff edge preventing small firms from accessing the workers they need," said Mr Cherry.
"This means having sensible transitional arrangements first, followed by the phased implementation of a new immigration system."
The FSB's survey of 1,236 of its members, reported in What Small Firms Want From Brexit, found that:
- 47% of respondents employed "mid-skilled" staff
- 72% of respondents had recruited their EU staff after they had already moved to the UK
- 95% of the respondents had no experience at all of the UK's current "points-based "system for obtaining work permits for non-EU citizens
- 56% of respondents were worried about applying any new immigration rules after Brexit, and
- As many as 13% might move their businesses abroad, cut back on their activities in the UK or close altogether.