A counterfeit clothing company in Leicester had workers sleeping on mattresses in the factory, investigators found.
The Gangmasters and Labour Abuse Authority (GLAA) checked 172 factory units in the city.
In one centre five workers were found to be living in their own workplace.
The probe comes after concerns were raised about working practices in the city's textile industry during lockdown.
GLAA director Daniel Scully leads Operation Tacit, which sees them work with the police, HMRC and other agencies to investigate issues around the workplace, including employers not paying minimum wage.
After executing search warrants in places where they "struggled to get co-operation", a number of "really old buildings" were found to be in "not great" condition, with one site used to counterfeit goods also housing its employees.
"It was clear some of the workers were sleeping in the factory," he said.
"That's not an environment where people should be overnight - it's not a residential place, it's full of paint, chemicals [and] screen-printing equipment.
"Where workers are being kept on the premises, that's really not a great situation for anyone to be in."
Mr Scully said they have been "working through a list" based on intelligence gathering and whistle-blowing, which has uncovered health and safety breaches, including some employers not implementing measures to tackle the coronavirus pandemic.
"We've also found some allegations of suppressed earnings, where workers are actually doing more hours than the books would show," he said.
"[There are] also some suggestions that money is being clawed back from workers, so on the paper records they will look like they've been paid the right amount, but some of that money has made it back to the company."
None of the factories were found to be carrying out "modern slavery", but Mr Scully said there was "definitely exploitative practices" being carried out across Leicester.
There was also evidence of firms being dissolved by the owners to avoid inspection before being reopened as a new business - a practice known as "phoenixing".
Though no businesses have yet been shut down by the GLAA, Mr Scully said there is the possibility criminal proceedings will be brought against some owners, adding recent action is "having an effect".
"There are more factories paying the minimum wage, [and] we know particularly now that as we've moved to different tactics, including executing warrants, that that will also cause factory owners who aren't compliant to think twice about the conditions that they're subjecting their workers to," he said.
Earlier this week, the co-founder of fashion chain Boohoo told MPs the company was working to "make Leicester right".
It came after the firm was accused of tolerating widespread abuses of employment law at some of its suppliers in the city.