A £1m donation to the National Portrait Gallery has been withdrawn because the potential donors are alleged to have fuelled the US opioid crisis.
The Sackler Trust, run by the family that owns Purdue Pharma, seller of prescription painkiller OxyContin, said the donation might "deflect" the gallery from its important work.
The Sackler family has "vigorously denied" the allegations against it.
The National Portrait Gallery said it supported the family's decision.
The Sackler Trust offered the money in 2016 to go towards the gallery's £35.5m redevelopment. The gallery had been mulling over whether to accept it.
In a joint statement, the Sackler Trust and the National Portrait Gallery said they had "jointly agreed not to proceed at this time" with the donation.
A spokesman for the Sackler family said: "The giving philosophy of the family has always been to actively support institutions while never getting in the way of their mission. It has become evident that recent reporting of allegations made against Sackler family members may cause this new donation to deflect the National Portrait Gallery from its important work.
"The allegations against family members are vigorously denied, but to avoid being a distraction for the [National Portrait Gallery], we have decided not to proceed at this time with the donation."
Purdue Pharma is facing lawsuits in the US alleging that it sold OxyContin as a drug with a low chance of causing addictions, despite knowing this was not true.
The US Centres for Disease Control and Prevention has said that opioids - a class of drug which includes everything from heroin to legal painkillers - were involved in almost 48,000 deaths in 2017.
President Donald Trump has called the US opioid epidemic a "national shame" and declared it a public health emergency.
The epidemic started with legally prescribed painkillers including Percocet and OxyContin. It intensified as these were diverted to the black market.
There has also been a sharp rise in the use of illegal opioids including heroin, while many street drugs are laced with powerful opioids such as Fentanyl, increasing the risk of overdose.
Will Gompertz, arts editor
Museum directors throughout the country might well have mixed feelings about the Sackler Trust's decision not to go ahead with a planned £1m gift to the National Portrait Gallery because it did not wish to "distract" the institution from its "important work".
On the one hand, they will be relieved that some clarity has been brought to the Sackler situation, which has become a very hot topic in the subsidised arts sector.
Should they or shouldn't they take the money? Is the reputational risk too high? Would other key supporters approve? Could there be a public backlash?
Right now, it would appear, there are too many downsides - probably for both parties - to accept a high-profile, million-pound-plus donation with the Sackler name attached.
On the other hand, they will be throwing their arms up in frustration.
Fundraising is very difficult at the best of times, but it is harder still when you are discounting one of the wealthiest and most generous philanthropic donors in the land.
The Sacker Trust is not one of thousands of similar foundations with huge sums to donate to the arts.
It is not even one of hundreds. It is one of a very few who have the reserves and commitment to consistently make large-scale gifts to arts organisations.
With it out of the game, at least temporarily, there are likely to be a few fundraising targets missed because of a Sackler-sized hole.
The Sackler family is one of the wealthiest families in the US.
The Sackler brothers - Raymond, who died in 2010, and Mortimer, who died in 2017 - set up Purdue Pharma in 1952.
The firm's fortunes rose with the launch of OxyContin in 1995 and the drug formed the basis of the Sackler family fortune.
The family still wholly owns the company, with about 20 members sharing in the money.
According to Forbes magazine, in 2016 the family was worth about $13bn (£10bn).
The company has paid hundreds of millions in the past to settle charges that it said OxyContin was safer and less addictive than it was.
In a lawsuit filed earlier this year, the Massachusetts Attorney General said dangerous opioid drugs were killing people across the state and that prescription medicines, which were supposed to protect health, were instead "ruining people's lives".
"Purdue Pharma created the epidemic and profited from it through a web of illegal deceit.
"At the top of Purdue, a small group of executives led the deception and pocketed millions of dollars."
The defendants were named as two companies and seventeen individuals who "engaged in a deadly, deceptive scheme to sell opioids in Massachusetts".
The members of the Sackler family named in the lawsuit were Richard Sackler, Beverly Sackler, David Sackler, Ilene Sackler Lefcourt, Jonathan Sackler, Kathe Sackler, Mortimer Sackler, and Theresa Sackler.
Campaigning group Culture Unstained welcomed Tuesday's announcement that Sackler Trust money would not go to the National Portrait Gallery.
"The Gallery's decision to reject a donation from those that profited from the opioid crisis is a powerful acknowledgement that some sources of funding cross a red line," it said.
The group hailed "renowned artist and former opioid addict" Nan Goldin, who it said "significantly escalated the pressure on the gallery after she threatened to pull out of a planned retrospective if it were to accept the donation".
"It was Nan Goldin and her supporters who forced the [National Portrait Gallery] to confront the ethical questions surrounding the Sacklers."
Many UK cultural institutions have benefited from donations from the Sackler Trust in the past, including the Serpentine's Sackler Gallery, the Victoria and Albert Museum in London and the National Gallery.
However, the BBC understands no UK institutions have applied for fresh funding from the trust since the allegations emerged in 2017.