Business Secretary Alok Sharma has admitted that "more money needs to go out faster" to businesses applying for emergency loans from the government.
He said 4,200 loans, worth a total of £800m, had been given to firms seeking cash to survive the coronavirus crisis.
However, that is just 1.4% of the 300,000 enquiries that are thought to have been made through the scheme.
Mr Sharma said banks were working "at pace" to issue the loans, which are backed by the government.
"They've got people working over the weekend talking to customers, making sure that they can process these loans as quickly as possible," he told BBC One's Andrew Marr show.
But the former Bank of England governor, Mervyn King, later expressed worries about how few businesses had got money through the scheme.
Speaking to Sophy Ridge on Sky News, Mr King said the survival of businesses through the crisis was key to a quick economic recovery.
"The economy will recover quickly only if we can keep the businesses that existed at the beginning of it still functioning and still able to pick up the reins when the epidemic is over," he said.
"If we find so few business loans being granted, something has gone wrong."
Mr King said it was a mistake to close local bank branches that could have been able to function - with social distancing - to respond to the queries and claims of business trying to apply for coronavirus business interruption loans.
Earlier this month, the government overhauled the scheme in response to claims that banks were taking advantage of the crisis.
Following criticism from firms and MPs, the government banned banks from asking company owners to guarantee loans with their own savings or property when borrowing up to £250,000.
It also removed a requirement that businesses must have been refused a loan on commercial terms in order to be eligible for the scheme after firms complained of facing interest rates of up to 30% and being asked to make unreasonable personal guarantees.
Mr Sharma told the BBC that the changes to the scheme would make it more attractive. But he said there would "no doubt" be economic repercussions following the lockdown measures.
"I think it is absolutely vital we put the support in now so that, when we come out the other side, business are able to start very quickly."