Welsh universities lack leadership and are not lifting the nation up the UK prosperity table, says the education minister.
Leighton Andrews said the governance of higher education had become "the last resting place of the crachach (elite)".
He said financial support would depend on ministers' objectives being met.
Cardiff University's Prof Teresa Rees said universities were doing their best "in challenging times".
Speaking at Cardiff University's School of City and Regional Planning in Cathays Park, Mr Andrews warned that he would "blunt and candid".
In his speech, the minister said: "I have begun to wonder whether the higher education sector in Wales actually wants the assembly government to have a higher education strategy, or whether it even believes that there is such a thing as a Welsh higher education sector.
"I am not alone in this view."
Mr Andrews warned: "The assembly government's education budget can no longer be a Christmas tree with presents for everyone. Money will follow One Wales' commitments and ministerial objectives.
"We have had more higher education institutions per head in Wales than any other part of the UK but have failed to break free from the bottom end of the UK growth and prosperity table."
The minister said Wales' higher education institutions were small compared with those in England.
He added that, for all their achievements, "they have had only a very limited transformative impact on our economy, and on our global presence and reputation".
"We are not having a high enough impact in terms of the quality and quantity of our research," he said.
"For too many in Wales, higher education remains a distant, and irrelevant activity, clouded in mystery."
Mr Andrews said he had been interested to learn recently that some members of university governing bodies had been appointed on the basis of a phone call, saying it was a question of "who you know, not what you know.
"It appears that HE governance in post-devolution Wales has become the last resting place of the crachach," he said.
"I take governance seriously".
He later told BBC Radio Wales: "The danger is we see a death by 1,000 cuts.
"There is a short window for the Higher Education sector to get its act together and that is probably only this financial year.
"And therefore if we do not see significant change, there is real danger of a reduction of in quality by simple salami slicing."
Reacting to Mr Andrews' speech, Prof Rees, pro-vice chancellor for research at Cardiff University, said Welsh universities had had "a bit of a battering in recent years".
She said: "We do have a funding gap compared with the institutions in England with whom we have to compete and indeed with the rest of the UK.
"I think we are doing our best in terms of teaching and learning and in terms of research and in terms of innovation and engagement with the community. But we are in challenging times."
'A very good job'
Prof Jonathan Osmond, the university's pro-vice chancellor for teaching and learning, disagreed with Mr Andrews' assertion that universities' governing bodies were the "last resting places for the crachach".
"That's not the way I would see it," he said.
"My experience is both at Cardiff University and also I'm on the board of trustees of the national museum so I have some appreciation of the different context but also the different roles that the governors can play.
"There are still ways in which we can make the appointment of governors more transparent, more widely accessible.
"But in my experience governors do a very good job. They are willing to criticise where appropriate in a constructive manner."