One in five British households have experienced slow broadband speeds in the last year, according to Which.
A similar number have had connection dropouts and router problems, the consumer rights group added.
It polled more than 1,900 customers across 12 providers, more than half of whom reported having problems with their broadband service or price.
Which said providers had "a long way to go" to meet customer expectations.
The survey looked at the biggest providers, including BT, Sky, TalkTalk, Virgin Media and Zen Internet. Together they serve about 90% of UK broadband customers.
It said TalkTalk customers were most likely to experience "very slow" speeds, with 31% of them reporting problems over the last 12 months.
They were followed by BT users, 23% of whom had issues. Sky came next, with 22% of its customers having complained.
When it came to connection dropouts, SSE was the worst performer with almost a quarter of its customers affected.
And Which said Virgin Media customers were the most likely to face router issues, and be left with no internet at all "for hours or even days at a time".
Overall, Virgin Media customers reported feeling most dissatisfied with their service, although their top complaint was about recent price rises, with some 47% reporting an increase in the last year.
Price was also the top complaint about BT, with 30% of customers reporting an increase.
A Virgin Media spokesman said: "We do everything we can to balance keeping our prices competitive while investing to meet customers' ever-increasing appetite for faster broadband and better content."
It comes despite new measures from Ofcom offering automatic compensation for slow repairs, missed appointments and delayed installations.
Alex Neill, Which's managing director of home products and services, said: "It's clear that too many households are still suffering from substandard broadband and our research shows that providers have a long way to go before they meet their customers' expectations.
"Customers who suffer from a shoddy broadband service will welcome new measures for automatic compensation, however providers should do more to stop these problems from happening in the first place."
Digital minister Margot James said: "Broadband has become a modern necessity, and we all know how frustrating it is when it doesn't work.
"We've introduced a raft of measures with Ofcom to help protect consumers, including receiving compensation when their service falls short, and making it easier for customers to switch provider if they're not getting what they've paid for and are entitled to."