The UK government has increased the relief fund for British overseas territories devastated by Hurricane Irma to £32m, Theresa May has said.
The announcement - increasing the fund from £12m - was made by the prime minister as she said the government had responded "swiftly" to the disaster.
Defence Secretary Sir Michael Fallon said the government was doing all it could to help people affected.
But Baroness Amos said it was felt the UK "did not respond" quickly enough.
The former UN under-secretary-general for humanitarian affairs said those in the overseas territories would want an "urgent and immediate" response from the UK.
"It's always the people on the ground who respond first. They are looking to their national governments but they are also looking outside for as much help to come as quickly as possible.
"We are now a couple of days in and I think people are feeling Britain did not respond quickly enough given we know that it is hurricane season."
The low-lying British territory of Turks and Caicos is still in the storm's path and evacuations have started.
Hurricane Irma has caused widespread destruction across the Caribbean, reducing buildings to rubble and leaving at least 10 people dead.
Foreign Office minister Sir Alan Duncan said British overseas territory Anguilla received the hurricane's "full blast" while the British Virgin Islands would need "extensive humanitarian assistance".
At least one death has been reported on Anguilla, according to local officials.
A third British territory, Montserrat, was "swiped" but the damage was not as bad as first thought, Sir Alan said.
Briton Emily Killhoury has lived on Tortola, the main island in the British Virgin Islands, with her husband Michael and their two children, aged nine and 10, for five years.
She told the BBC her family bunkered down in a closet when the storm hit.
"Our downstairs doors suddenly blew out which was terrifying. We just stayed hiding," she said.
"We eventually emerged at about 7pm to see total devastation. Everybody is shocked but trying to be practical."
Simon Cross, 32, who also lives on Tortola, said it had gone from being a "lush green tropical island to almost like there's been a fire".
"There are boats upside down all twisted up. You can see corrugated sheeting littering all of the mountain.
"It's unclear as to how many people are trapped. I can't get hold of any of my friends.
"The island needs additional resources from the UK urgently."
Royal Navy ship, the RFA Mounts Bay, is in the region and a second British ship, HMS Ocean, is also being sent, but is not expected to arrive for another two weeks.
Speaking after a meeting of the government's Cobra emergency committee, Sir Michael said the UK's taskforce would help with relief efforts, such as restoring clean water, providing medical assistance and reconstruction work.
But Josephine Gumbs-Conner, a barrister from Anguilla, said the UK's preparations for, and response to, the storm had been "sorely lacking".
She told BBC Radio 4's Today programme the UK government should have "done what the French did in St Martin - who made sure that they had military on the ground so that the response given is timely".
International Development Secretary Priti Patel said "world-leading humanitarian experts" as well as 200 shelter kits had been sent to the region as part of the UK's response.
The Queen said she and Prince Philip have been shocked and saddened by reports of the devastation.
In a message to the governor-general of Antigua and Barbuda, she said: "Our thoughts and prayers are with all those whose homes and livelihoods have been destroyed or adversely affected by this terrible storm."
Thousands of British tourists are believed to be holidaying in the Caribbean, the travel association Abta said.
The Foreign Office had warned Britons to evacuate the area, as the most powerful Atlantic storm in a decade approached.
Fears are growing for pregnant Briton Afiya Frank, 27, and her sister Asha Frank, 29, who were preparing for the storm in Barbuda but have not been heard from since Tuesday night.
Their aunt, Ruth Bolton, told BBC Radio Suffolk the pair had "gone completely silent" since they last messaged on WhatsApp at about 21:00 GMT on Tuesday.
She said Afiya had been due to return to Suffolk to give birth.
Many expats and tourists have been left stranded as airlines were forced to ground or divert flights.
British Airways evacuated 326 passengers from Antigua on Tuesday and has managed to rebook many others across the Caribbean islands on to flights out with alternative airlines.
Virgin Atlantic said it had scheduled a relief flight "loaded with essential items" to help the recovery effort, including blankets and bottled water, to arrive in Antigua on Thursday.
Many British tourists staying at resorts in the Dominican Republic, where a hurricane warning is still in place, are being evacuated from coastal areas and moved to temporary shelters.
Andrea Fowkes Smith, from Surrey, told the BBC that part of the roof had fallen off the hotel where she was sheltering in Punta Cana.
"We have not been evacuated from our hotel but have just been moved to the steak house as our room was on the third floor," she said.
"They say we should all stay on the ground," she added. "It's very strong winds and rain."
Officials in the US have started evacuations of tourists and residents from Florida Keys as the hurricane approaches.
Flights to and from several airports in Florida were being suspended, while Orlando's international airport said commercial flights would stop from 17:00 local time on Saturday.
A state of emergency has been declared for Florida, Puerto Rico and Cuba.
Are you in the region? Are you a holidaymaker unable to get a flight home or a resident who has been preparing for Hurricane Irma? If it is safe for you to do so, share your experiences by emailing email@example.com.
Please include a contact number if you are willing to speak to a BBC journalist. You can also contact us in the following ways: