Spending on new social housing and road maintenance face severe cuts under the assembly government's draft budget.
The housing capital budget will fall from £273m this year to £206m by 2013-14.
Community Housing Cymru warned new social housing in Wales will "take a hit" unless new funding can be found in the next six months.
There will also be a steep cut in the road maintenance budget. Ministers said "difficult choices" had to be made.
The assembly government needs to find 40% savings in spending on Welsh infrastructure over the next four years.
The Institute of Directors (IOD) warns that Wales' road network is already "creaking" and they are disappointed that the amount spent on transport will be cut.
On housing, the assembly government has already indicated that it intends to set up a new Welsh Housing Investment Trust to try and meet some of the shortfall by attracting investment from outside sources.
Responding to the figures, the chief executive of Community Housing Cymru, Nick Bennett said: "During the past three years, we've built up the capacity through doubling the building of social housing units in Wales to the point that in 2009, we saw the first increase in the overall stock of social housing in Wales since the introduction of the right to buy in 1981.
"Our only way forward now is to embrace innovation, either through the Welsh Housing Investment Trust or an alternative measure.
"Unless we can find an innovative set of measures that can be implemented over the next six months, then the building of new social housing in Wales is clearly going to take a hit."
The assembly government's budget document points out that within housing they are maintaining budgets to help bring current and former local authority housing stock up to Welsh Housing Quality Standard.
Meanwhile, according to the draft budget, the amount earmarked to spent on trunk road maintenance by the assembly government will also fall sharply in the coming years.
The budget line for improving and maintaining the trunk road network will fall from £75.3m to just £32.7m over the next three years.
The head of the IOD in Wales, Robert Lloyd Griffiths said: "It's vital that Welsh Assembly Government ministers now make wise spending decisions and do all in their power to preserve vital infrastructure investment.
"Key parts of our infrastructure are creaking, particularly the roads network.
"It's disappointing that the assembly government has chosen to cut significantly the already inadequate sums it spends on transport. This does not seem to sit comfortably with their avowed position on encouraging economic growth.
"Of course, cuts in spending should not mean cuts in output of the public sector. Nothing should divert attention from the need to improve the efficiency of our spending in the public sector in Wales.
"Spending has increased hugely since devolution and it is time for the taxpayer to get a proper return on that spending. Reductions in input should not be synonymous with reductions in output."
The assembly government's budget document states: "We will ensure sufficient funding is available to promote a safer environment for those who use the local and regional networks.
"However the department has had to accommodate a significant reduction in its capital budget allocation and this will mean that the planned delivery schedule and profiling of certain road and rail schemes will have to be reviewed."
Assembly Finance Minister Jane Hutt said ministers were on record as stating their opposition to the "speed and depth of cuts imposed by the UK Government at this crucial phase of recovery from recession".
She added that "difficult choices" had to be made and they had "worked hard to reduce the duration and impact of any reductions".
"However, having been dealt this hand, we are determined to do what it takes to stand up for the people of Wales and continue to provide the right support for those who need it most."