Boris Johnson's chief adviser, Dominic Cummings, has attended meetings of the scientific body shaping the government's coronavirus response.
Downing Street denied a Guardian report he was a member of the Scientific Advisory Group for Emergencies (Sage).
The committee, whose membership is not public, gives independent advice.
A senior Tory has called for "transparency", while Labour said Mr Cummings' attendance raised "significant questions".
The Guardian reported that Mr Cummings and a data scientist who worked with him on the Vote Leave campaign during the Brexit referendum, Ben Warner, were among 23 people at a Sage meeting on 23 March.
That was the same day the prime minister announced the nationwide lockdown, bringing in strict new measures to tackle coronavirus.
A No 10 spokesman said Mr Cummings and Mr Warner had attended or listened in to Sage meetings in order to better understand the scientific debate around coronavirus.
He said they "occasionally" asked questions or offered help when "scientists mention problems in Whitehall", adding that others "also listen to meetings without being on, or a member of, Sage".
"The scientists on Sage are among the most eminent in their fields," the spokesman said. "It is factually wrong and damaging to sensible public debate to imply their advice is affected by government advisers listening to discussions."
He added: "'Public confidence in the media has collapsed during this emergency partly because of ludicrous stories such as this."
Sage is a panel of medical and scientific experts, chaired by the UK's chief scientific officer Sir Patrick Vallance, that provides independent advice to the government during a crisis.
The make-up of the committee has been kept secret - although individual members can disclose they are part of the group - and its advice has been given to ministers but has not been made public.
NHS England's medical director Stephen Powis, who is one of the few publicly known members of Sage, told BBC Radio 4's Today programme: "I have been confident that what happens at Sage is a scientific discussion involving the scientists and the experts who are members of Sage."
Conservative MP David Davis said transparency was important and pointed out the monetary policy committee published details of its membership and advice.
The former Brexit secretary tweeted: "We should publish the membership of Sage: remove any non-scientist members: publish their advice in full: and publish dissenting opinions with the advice."
His colleague Damian Green, who was Cabinet Office Minister in Theresa May's government, said he "would be much more worried" if senior people from Number 10 were not sitting in on these meetings, to ask questions and get the tone of the meeting.
But Labour's shadow health secretary Jonathan Ashworth said: "He is a political adviser, not a medical or scientific expert. If the public are to have confidence in Sage, the government must make clear Dominic Cummings can no longer participate or attend."
"The concern is that political advisers have influenced the debate," he told BBC Radio 4's Today programme.
Mr Ashworth also called on the government to be "entirely transparent" about its decisions and to publish the minutes of Sage meetings.
Sir David King, the government's former chief scientific adviser, said he was concerned by Mr Cummings' attendance because if he participated in the discussion "this must compromise the independence" of the group.
He told BBC Newsnight: "What we don't know is the influence he plays in taking what he interprets from those meetings back to the prime minister."
BBC Newsnight policy editor Lewis Goodall said the the disclosure about Mr Cummings would contribute "to further calls for Sage and the advice that it gives to the government to be published and its membership to be fully disclosed".
On Friday, England's chief medical officer Prof Chris Whitty, a member of the body, told MPs he had no objection "in principle" to details of the membership being released.
Mr Cummings has recently returned to work after self-isolating with symptoms of coronavirus. His wife, journalist Mary Wakefield, described how he collapsed and was bed-ridden for 10 days.