Investigators have found that in 2007 the German Intelligence Service (BND) destroyed files of 250 BND employees who had been in the Nazi SS or Gestapo.
The BND confirmed the loss, calling it "regrettable and annoying".
Four independent historians are investigating the BND's old links with the Nazis. They say some of the missing papers concern suspected war criminals.
The historians did not allege a deliberate cover-up, but they urged the BND not to destroy any more files.
They said the BND should consult them before shredding any more documents, and called for a full investigation into the 2007 incident.
One of the historians, Dresden Professor Klaus-Dietmar Henke, said "it is not the case that the BND somehow deliberately destroyed the files of all those with Nazi links".
According to the BND, the destroyed files relate to about 2% of the archive currently being investigated by the historians. The files were destroyed because they were not deemed to be worth keeping, the BND said.
The investigation into the BND's Nazi roots began in February.
Reinhard Gehlen, who set up the BND after World War II, spied on the Soviet Union for the Nazis during the war.
About 10% of BND recruits during the Cold War had previously served in the SS, the German news website Spiegel reports.
Spiegel says the BND told it that in the 1990s documents about a top wanted Nazi, former SS Capt Alois Brunner, were destroyed.