Matt Hancock has said he is concerned about working practices in some clothing factories in Leicester, which is under local lockdown.
Speaking on the BBC's Andrew Marr Show, the health secretary said he was "very worried about the employment practices in some factories" in the city.
Leicester was put on local lockdown on Monday after a spike in Covid-19 cases.
Mr Hancock told Sky's Sophy Ridge he was "very worried" about the number of infections in Leicester.
He confirmed the government has already shut down businesses during the coronavirus pandemic.
Asked if there was a link between employment practices in Leicester and the outbreak there, Mr Hancock described guidance for employers as "statutory guidance" backed up by fines.
"There are clearly some problems that have been under the radar in Leicester that need action," he told the BBC.
Speaking on Sophy Ridge on Sunday, Mr Hancock said: "We've seen outbreaks in clothing factories and in food factories and there are some quite significant concerns about some of the employment practices in some of the clothing factories in Leicester, they are important problems to deal with.
"We also have the authority to shut down the business if it doesn't follow the guidance."
On Thursday, online fashion retailer Boohoo defended its business practices after a workers' rights group said staff at Leicester factories that supply the group were at risk of coronavirus.
Labour Behind the Label said workers were "being forced to come into work while sick with COVID-19".
Boohoo said it had "terminated relationships" with factories over the treatment of workers.
In a statement, the group said: "The Boohoo group will not tolerate any incidence of non-compliance especially in relation to the treatment of workers within our supply chain."
The group said it would investigate the allegations and take any necessary action.
A HSE spokesperson previously said: "In Leicester we are actively investigating three textile businesses, have recently contacted 17 and undertaken three site visits.
"Enforcement action is being taken at one of these sites and further spot inspections will take place in the area in the coming days and weeks."