The men suspected of attacking women in Cologne on New Year's Eve were "almost exclusively" from a migration background, mainly North African and Arab, an official report says.
Cologne police also made "serious mistakes" in not calling reinforcements and the way they informed the public.
More than 500 criminal complaints were filed, 40% alleging sexual assault.
Apparent retaliatory attacks in Cologne on Sunday were condemned by the government as inexcusable.
The Pope meanwhile said Europe's "humanistic spirit" risked being undermined.
The "immense influx" of migrants was causing problems, he said, but the continent had the means to strike a balance between protecting its citizens and helping migrants.
North Rhine-Westphalia state's interior minister Ralf Jaeger said that recent arrivals in Germany were among the suspects in the attacks, which took place in central Cologne, in the area of the cathedral and the main railway station.
Nineteen individuals are currently under investigation by the state police in connection with the attacks, NRW's interior ministry says in a report (in German), none of them German nationals.
Those 19 suspects include 14 men from Morocco and Algeria. Ten of the suspects are asylum seekers, nine of whom arrived in Germany after September 2015.
The other nine are possibly in Germany illegally, the interior ministry says.
The scale of the assaults on women in Cologne and other German cities has shocked Germany.
Mr Jaeger is himself under political pressure. On Friday he fired Cologne's police chief, Wolfgang Albers.
Addressing state MPs on Monday, Mr Jaeger criticised police for not calling for reinforcements on the night, and also for the way they informed the public about the investigation in the days after the events.
His report details how a group of around 1,000 men of North African and Arabic origin gathered on 31 December. Smaller groups formed, surrounding women, then threatening and attacking them, he said.
These groups were predominately made up of North African men who had travelled to Cologne from different cities.
"After the intoxication with drugs and alcohol came violence," said Mr Jaeger. "It culminated in the acting out of fantasies of sexual omnipotence. That must be severely punished."
Around 1.1 million asylum seekers arrived in Germany in 2015.
Chancellor Angela Merkel's immigration policy has come under criticism since the attacks.
Riot police used water cannon to disperse anti-migrant protesters in Cologne on Saturday.
On Sunday evening, a group of around 20 people attacked six Pakistanis, two of whom had to be treated in hospital.
Two Syrian men were attacked in two separate incidents. Three men from Guinea were assaulted in a fourth attack, police said.
Crime division chief Norbert Wagner said police were investigating whether the attacks were co-ordinated on social media.
Justice Minister Heiko Maas condemned those attacks, saying: "As abominable as the crimes in Cologne and other cities were, one thing remains clear: there is no justification for blanket agitation against foreigners."
He said it appeared that some people "appear just to have been waiting for the events of Cologne".
Mrs Merkel's spokesman Steffen Seibert said "nothing excuses" retaliatory assaults on immigrants.
Monday's report into the attacks in Cologne says that the combination of group sexual violence with robbery had not previously been seen in Germany.
It notes that similar crimes took place in other parts of Germany on 31 December, including in Hamburg.
The report describes a modus operandi known as "taharrush gamea" in Arabic, meaning group sexual harassment in crowds, and compares it to incidents reported in Cairo's Tahrir Square at the time of the Egyptian revolution.
A joint federal and state working group has been set up to examine the phenomenon and how to combat it.