The Museums Association has voiced concern over what it sees as government "interference" into how museums present Britain's imperial history.
"It is not for ministers to dictate what constitutes a legitimate subject for investigation," the body said.
Culture Secretary Oliver Dowden held a meeting on Tuesday with representatives from leading heritage organisations.
The government said: "We absolutely support the independence of museums and heritage organisations."
Writing on Twitter after Tuesday's virtual meeting, Mr Dowden described it as a "very productive conversation about protecting heritage for future generations".
He said a working group would be set up and guidelines published to put the government's "retain and explain" policy into practice.
Very productive conversation about protecting heritage for future generations— Oliver Dowden (@OliverDowden) February 23, 2021
Agreed to jointly set up a working group & produce guidelines to put the govt's retain and explain policy into practice, so that more people can engage with our shared past pic.twitter.com/TYkWVLpttd
The toppling of slave trader Edward Colston's statue in Bristol last year, along with other campaigns, have led to a heated debate about whether and how historical objects linked to Britain's colonial past should be displayed.
In September, culture minister Matt Warman told MPs the UK's heritage should not be removed from view, "however contentious" it might be.
He said an effort should be made to "contextualise or reinterpret" memorials and artefacts "in a way that enables the public to learn about them in their entirety".
The National Trust was among the organisations at Tuesday's virtual discussion, and said it had been "helpful and productive". Historic England, which was also represented, said it would work with the government "to help people understand on a deeper level the many layers of our shared past".
Independence 'not at issue'
Sir Ian Blatchford, head of the Science Museum Group and chair of the National Museum Directors' Council, said: "The meeting was a timely exchange about the importance of telling a balanced and well-researched history of Britain, and a moment to dispel some of the mischief-making around this event.
"The independence of museums is not at issue, and UK museums remain as committed as ever to the brilliant and diverse storytelling that our audiences so value."
According to Museums Association director Sharon Heal, however, the government's "perceived interference" on matters relating to empire and slavery is a subject for concern.
"It is both a hallmark of a democratic society and a cornerstone of museum ethics that our sector should operate at arms-length from the government," she said.
'Climate of fear'
"It is vital that the government consults widely with the sector - including with the Museums Association - before proceeding with its proposal to produce guidelines on 'contested heritage'.
"We are particularly concerned that a climate of fear has been created amongst museums and museum staff, especially those working on subjects relating to Britain's imperial past."
Around 15,000 workers from the UK's heritage sector are members of the Museums Association.
In response, a spokesperson for the Department for Digital, Culture, Media and Sport said: "We have been crystal clear that we absolutely support the independence of museums and heritage organisations and the National Museums Directors' Council has acknowledged this.
"The meeting this week with Arm's Length Bodies and other stakeholders was the start of a conversation about how we work together to protect our heritage for future generations. Public funded bodies recognise that they have a responsibility to act impartially, in line with their status."
The Imperial War Museum, National Portrait Gallery, British Film Institute, British Museum and Horniman Museum are among other organisations understood to have participated in Tuesday's discussion.