More than 50 MPs and peers have written to the home secretary urging her to do more to protect UK garment factory workers from exploitation.
It follows reports of staff at factories in Leicester being underpaid and unprotected from Covid-19.
Fast fashion brands Boohoo and Quiz were both accused of using unethical suppliers in the city and have since vowed to investigate.
The government said exploitation for commercial gain was "despicable".
The letter - which was also signed by investors, charities and retailers such as Asda and Asos - said concerns around unethical use of labour in the UK's garment industry had been raised "multiple times" in the last five years by academics, retailers and MPs, but little had been done.
It said "thousands more" could be exploited without stronger government action.
"The public want to know that the clothes they buy have been made by workers who are respected, valued and protected by the law," said Helen Dickinson OBE, boss of the British Retail Consortium, which coordinated the letter.
Last week Quiz said it had suspended a supplier after reports that a factory in Leicester offered a worker just £3 an hour to make its clothes.
The national minimum wage for people over 25 years-old is £8.72 an hour.
Rival retailer Boohoo was similarly accused of using a factory that underpaid workers, while also doing nothing to stop the spread of coronavirus.
The letter urged Home Secretary Priti Patel to bring in a new licensing scheme for garment factories that would:
- Protect workers from forced labour, debt bondage and mistreatment, as well as ensuring the payment of the National Minimum Wage and holiday pay
- Prevent rogue businesses from undercutting compliant manufacturers
- And encourage retailers to source their clothing from the UK, supporting the development of an "ethical, world-leading industry".
On Friday, Boohoo boss John Lyttle wrote to Ms Patel urging her to adopt the proposals.
"We're taking action to investigate allegations of malpractice in our supply chain and we ask government to take action too," he said.
Both Boohoo and Quiz have said the claims made about their suppliers - if true - are "totally unacceptable" and have promised to take action.
'Free from exploitation'
The National Crime Agency has also confirmed it is investigating Leicester's textile industry over allegations of exploitation, although it did not comment on Boohoo or Quiz specifically.
Minister for Safeguarding, Victoria Atkins, said: "Exploiting vulnerable people for commercial gain is despicable and this government will not stand for it.
"We expect all companies implicated in these allegations to conduct a full and thorough investigation to ensure that their supply chains are free from labour exploitation.
"We have liaised with relevant agencies regarding alleged working practices at garment factories in Leicester. We await the results of these investigations."