The European Court of Human Rights says Russia has failed to explain why it kept key files secret when it investigated the 1940 Katyn massacre of more than 20,000 Polish war prisoners.
Russia failed to comply with a human rights obligation to provide evidence, the Strasbourg judges ruled.
But they also said the court had no authority to rule on the inquiry.
Soviet Russia only admitted in 1990 that its forces, and not the Nazis, had carried out the atrocity.
The massacre in Katyn forest, by the Soviet secret police (NKVD), took place before the European Convention on Human Rights was signed.
The court said it did not have competence to rule on Russia's handling of the Katyn investigation because too much time had elapsed between the massacre and the entry into force of the Convention.
The court's examination came after 15 relatives of the victims claimed that Russia had failed to carry out an adequate investigation.
They said Moscow had prevented them from finding out the truth about the killings in western Russia.
The Strasbourg judges dismissed the applicants' claim for just satisfaction in the case, known as Janowiec v Russia.
But the judges also said it must be assumed that the applicants' relatives had been killed at Katyn, as they had heard nothing since 1940 from the men who disappeared.
Moscow started a criminal investigation into the killings in 1990, but it was discontinued in 2004 on the orders of the Russian chief military prosecutor's office.
Files about that decision remain classified, and the Polish claimants have not had access to it or any other information about the investigation.
The court judgment on Monday - which is final - said states party to the Convention were "obliged to comply with its requests for evidence and found that Russia, in refusing to submit a key procedural decision which remained classified, had failed to comply with that obligation". They added that "the Russian courts had not conducted a substantive analysis of the reasons for maintaining the classified status".
No-one has ever been convicted in connection with the massacre, with Russian prosecutors arguing that those responsible are now dead.
However, in 2010 Russia's parliament issued a statement saying more work needed to be done in "verifying the lists of victims... and uncovering the circumstances of the tragedy".
The parliament also said the killings had been carried out on Soviet leader Joseph Stalin's orders.
In 2010, Russia also published online six once-secret files on the mass killings. The documents had until then only been available to researchers.
Poland has repeatedly demanded that Russia open all its files on Katyn.
Polish newspapers deplored Monday's court ruling. A headline in Gazeta Wyborcza said "Strasbourg tribunal washes its hands of Katyn crime", Rzeczpospolita said "Strasbourg: Polish defeat in Katyn case" and Nasz Dziennik said "There will be no justice for the Katyn victims".
The killings were carried out by the NKVD in April and May 1940 in the Katyn forest near the city of Smolensk, and also near the village of Mednoye, in the Tver region, and the village of Pyatykhatky, in what was then Soviet Ukraine.
The victims were members of the Polish elite, arrested after the USSR invaded and annexed eastern Poland in 1939.
They were shot on the recommendation of NKVD head Lavrenty Beria, according to one of the published secret files.
In a letter to Stalin dated 5 March 1940, Beria says the Polish prisoners of war should be executed and refers to them as "steadfast, incorrigible enemies of Soviet power".
"Each of them is just waiting for liberation so as to actively join the struggle against Soviet power," it says.
The letter bears Stalin's signature in blue pencil, with the comment: "In favour."
In April 2010, Polish President Lech Kaczynski and more than 90 other government officials were killed when their plane crashed as it was trying to land at Smolensk airport to attend an event marking the Katyn massacre.