The UK government says it is "disappointed" after the European Court of Justice said the "indiscriminate" collection of data was against EU law.
EU judges said communications data could only be retained if it was used to fight serious crime.
Its verdict came after a legal challenge to the UK government's surveillance legislation.
The challenge was initially championed by Brexit Secretary David Davis, who was then a backbench Conservative MP.
The Lib Dems said the ruling proved the government had "overstepped the mark" with its Data Retention and Investigatory Powers Act, branded the "snoopers' charter" by critics, which requires communications companies to retain data for 12 months.
The ECJ has ruled that a "general and indiscriminate retention" of data is against EU law and can only be done under certain conditions and "solely for the purpose of fighting serious crime".
Its ruling confirms a preliminary verdict in July. The case now returns to the UK Court of Appeal, which had referred the case to the ECJ for clarification.
Mr Davis, who had long campaigned on civil liberties issues, left the case after Theresa May appointed him to her cabinet in July.
Tom Watson, Labour's deputy leader, who is one of those bringing the case, said: "This ruling shows it's counter-productive to rush new laws through Parliament without a proper scrutiny."
The Home Office said it would be putting forward "robust arguments" to the Court of Appeal.
It added: "Given the importance of communications data to preventing and detecting crime, we will ensure plans are in place so that the police and other public authorities can continue to acquire such data in a way that is consistent with EU law and our obligation to protect the public."
Investigatory Powers Act
- Replaces the Regulation of Investigatory Power Act, which expires at the end of December
- The government says it gives police and the intelligence agencies the tools to keep people safe and addressing "ongoing capability gaps"
- But critics have dubbed it a "snoopers' charter", raising civil liberties concerns
- Among the measures is the requirement for communications companies - like broadband or mobile phone providers - to hold a year's worth of communications data
- Under existing law, agencies can already ask firms to start collecting this data - but they can't access historic information because companies don't keep it
The government said it would not make any changes until the Court of Appeal had ruled on the legal challenge to the legislation.
The term 'communications data' refers to details such as the time and date of a phone call being made or an email being sent - but not its contents.
The Data Retention and Investigatory Powers Act expires at the end of December, and new legislation, the Investigatory Powers Act, is in place.
Campaign group Liberty, which represented Mr Watson, said: "Today's judgment upholds the rights of ordinary British people not to have their personal lives spied on without good reason or an independent warrant.
"The government must now make urgent changes to the Investigatory Powers Act to comply with this."