The Sackler Trust has suspended new charitable donations in the UK amid claims the Sackler family fortune is linked to the opioid crisis in the US.
The trust said it rejected the claims, but said the row had become a distraction for the groups it supports.
Last week, the National Portrait Gallery become the first major art institution to give up a grant from the Sackler family.
Other organisations have also shunned Sackler money, including the Tate.
Since 2010 the Sackler Trust has committed £60m to a range of causes.
The Trust is one of many philanthropic organisations funded by the Sackler family.
The negative publicity the family has received, due to its connections with the US company Purdue Pharma and its controversial opioid painkiller OxyContin, has led to mounting pressure on museums and galleries not to accept its money.
In a statement, Dame Theresa Sackler, chair of The Sackler Trust, said: "I am deeply saddened by the addiction crisis in America and support the actions Purdue Pharma is taking to help tackle the situation, whilst still rejecting the false allegations made against the company and several members of the Sackler family.
"The Trustees of the Sackler Trust have taken the difficult decision to temporarily pause all new philanthropic giving, while still honouring existing commitments."
The Anglo-American billionaire dynasty are prolific philanthropists and some of Britain's best-known art galleries, museums, theatres and universities have benefited from their generosity.
But behind the money is a firm called Purdue Pharma, a US company owned by many of the Sacklers, which makes opioids - a class of drugs linked to the deaths of thousands of Americans.
In 2016, the Sackler Trust offered the National Portrait Gallery a grant worth £1m to go towards the gallery's £35.5m redevelopment. The gallery had been mulling over whether to accept it, but last week accepted the Trust's decision to withdraw the offer.
Many other UK cultural institutions have benefited from donations from the Sackler Trust in the past, including the Serpentine's Sackler Gallery, the Victoria and Albert Museum in London and the National Gallery.
In New York, the Solomon R Guggenheim Museum, which has received a total of $9m (£6.8m) from the family, said it did not plan to accept any more gifts from the family.
From respected to rejected
Analysis by Will Gompertz, BBC Arts Editor
The Sackler Trust has gone from being a respected and generous philanthropic donor to the arts to being something of a headache for those it supports.
Arts bosses and their trustees have been weighing up if the financial benefit of a Sackler gift is worth the reputational risk (and associated hassle that comes with it).
The National Portrait Gallery, and the Tate have both decided it is not.
For those institutions who were still unsure what to do, the announcement today by the Sackler Trust that it will "pause" all donations forthwith has made the decision for them.