River and fishing groups have rounded on Wales' environmental regulator Natural Resources Wales, branding it "no longer fit for purpose".
Budget cuts have meant it is failing in its duty to protect and improve fisheries, the charities claim.
Levels of pollution, particularly from farms, are "out of control", warned Afonydd Cymru and the Angling Trust.
Ministers said despite grant reductions it had not impacted on NRW's "ability to deliver their statutory duties".
Meanwhile, NRW admitted it faced "challenges" around pollution but said the organisation was trying to work differently "and work smarter".
NRW was formed in 2013, by merging the Countryside Council for Wales, the Environment Agency and the Forestry Commission, designed to be a one-stop shop for the environment.
One of its duties is to monitor the health of Wales' rivers, taking action against people or companies who pollute them.
Soil, fertiliser and slurry from farms as well as waste from industry can find its way into rivers, killing fish and the aquatic insects they feed on.
Figures show NRW received 6,886 reports of water pollution between April 2013 and December 2016.
About 60% were investigated, but there were just 41 prosecutions and 10 civil sanctions - which amounts to less than 1%.
Mark Lloyd, chief executive of the Angling Trust and Fish Legal, said the regulator needed to take a much tougher stance but did not have "the resources to investigate properly".
The NRW argues that it is misleading to equate effectiveness with the numbers of prosecutions, saying it takes enforcement action "where appropriate".
NRW has seen its budget squeezed repeatedly by Welsh Government.
Its total revenue and capital income is forecast to fall by £20m over three years, assuming cuts of 5% a year.
"They've lost a lot of their frontline staff and are unable to take concerted action," Mr Lloyd claimed.
"The organisation is unwieldy, too bureaucratic and they don't seem to have a strategy - I just think they're failing in their duty to protect the environment."
After a severe pollution incident killed thousands of fish on the River Teifi in Ceredigion in December 2016, Mr Lloyd wrote to Environment Secretary Lesley Griffiths to outline his concerns.
This was "by no means an isolated case; there have been a number of similar high profile pollution incidents in Wales in recent years," he said, with "the most important rivers" for fishing and biodiversity affected.
He claimed it highlighted a "national failure of Welsh Government" to tackle the problem.
Dr Stephen Marsh-Smith, chief executive of Afonydd Cymru, the umbrella body representing Wales' six rivers trusts, said he agreed with the comments made in the letter.
"We've seen terrible disasters in Wales, not just spot pollutions but the general diffuse pollution and the organic content that's going into rivers and really degrading them badly," he said.
The impact on fish stocks had been "devastating", he said and that had ramifications for the rural economy as well as the environment.
So could there be lessons for Wales from a project undertaken just across the border in Herefordshire?
There, England's Environment Agency is taking "a really proactive approach", according to Dr Marsh-Smith, partnering with the Wye and Usk Foundation.
Farms at risk of enforcement action for polluting the River Arrow, a tributary of the Wye, are identified and visited by the regulator.
They are then signposted to one of six Wye and Usk Foundation farm officers who can offer free, confidential advice about measures to prevent the problem.
Kate Adams, head of land use at the charity said the scheme was working "very well" and they had seen "localised improvements to water quality and an increase in river trout numbers".
She is working with Welsh Water to bring a similar project to west Wales.
The "key difference in England", according to Dr Marsh-Smith is "a regulator that is doing its main job, which is regulating - and it's getting on with that".
"We need that in Wales - urgently."
Pollution incidents from farms is not increasing significantly year on year but the NRW still regards it as too high and in need of reduction.
It also says the vast majority of pollution reports were very small and had limited impact on watercourses.
Huwel Manley, NRW operations manager, accepted the organisation was facing "a challenge around pollution at the moment".
"It has become serious over the latter few years, and is an issue very much here in west Wales," he said.
"We do have a system where we provide advice to landowners, we do take enforcement in various forms, around improving facilities on farms to formal cautions and in some cases we do take to court."
Mr Manley said if incidents of agricultural pollution were isolated in the figures, the percentage facing enforcement action would rise to around 6-7%.
But he said sanctions were only "one of the tools in the box" and the organisation was trying to work proactively with farmers, an approach supported by Welsh Government.
"Like the whole of the public sector we're under pressure with cutbacks and naturally there have been some reductions (in staff)," he added.
He emphasised there was a need to work in partnership with the farming industry and charities to help tackle the issue of pollution on Welsh rivers.