Controversy over a £225m Swansea city deal flagship project could "cause a loss of confidence", an independent review into the programme has warned.
The proposed "wellness village" in Llanelli aims to bring together health facilities and life sciences research.
Five staff have been suspended as Swansea University holds an investigation related to the project.
The chairman of the city deal joint committee said it would consider the report at the next meeting.
The independent review said as yet it had not heard evidence that undermine its business fundamentals.
And the report said it "certainly" should not undermine delivery of the £1.3bn city deal as a whole.
What is the wellness village?
- It would cost £225m and be built on former industrial land at Delta Lakes close to the Llanelli coast
- It would bring together a university life sciences institute, retirement and care village, an eco-park, community health hub and rehabilitation centre, as well as "wellness hotel" with spa - to try to tap into the emerging wellness tourism sector
- It is one of the key projects in the Swansea city deal, which aims to create more than 9,000 jobs over 15 years in technology and science
- Initial ground works started at the site in 2017 and Carmarthenshire planners supported the project in January 2019.
The city deal brings together four councils, two universities, two health boards and private business - backed up by £241m UK and Welsh government support.
The rapid review - by Actica Consulting - was ordered by both the Welsh and UK governments into how the city deal was being managed and implemented.
It recognised that governance issues involving the wellness village have been given a high priority by the city deal and that "the restoration of public confidence may take some time".
It also said it was important the city deal kept its cohesion, warning that a combination of concerns over funding and of the "much-publicised concerns on the wellness village could cause a loss of confidence within the region".
It added that the loss of a key local authority partner could prove severely damaging to confidence of private partners. "In this context, real progress must be demonstrated and we suggest that the time for exchange of emails and revised business cases has passed," it added.
Meanwhile, an internal review by the four councils involved in the city deal - seen by BBC Wales - says the investigation at Swansea University and its links to the wellness village were having a "detrimental impact" and "eroding trust" within the city deal partnership.
However, the partners - which include Pembrokeshire, Neath Port Talbot, Carmarthenshire and Swansea councils - remain committed to delivering the plan.
This review concludes there is a lack of certainty over funding and that the business cases for the projects in the deal lack clarity, which the report argues is "undermining confidence" in the process in the eyes of the UK and Welsh governments.
The councils have asked the two governments to "hold" the business case for the wellness village.
The report says a number of issues have been raised by the suspension of staff at Swansea University which include business links between individuals and firms involved in the wellness village and whether personal interests had been declared.
The Actica report says that the approval of at least some city deal projects this year was "critical both financially and to build confidence".
It found that two projects could progress and that their business cases were "fit for purpose".
These are the Yr Egin creative digital cluster in Carmarthen, around the new S4C headquarters, and the £168.2m City and Waterfront digital district in Swansea, which includes a 3,500-seat indoor arena.
Councillor Rob Stewart, chairman of the city deal joint committee, said: "This review was carried out alongside the UK and Welsh government's independent review of the city deal programme and sought to assure that it will deliver full economic benefits for the region.
"The findings and recommendations of the internal review will be formally considered by the SBCD joint committee at the next meeting.
"Looking to ensure governance is as robust as possible reflects that we're still in the very early stages of a 15-year programme, but we're ready to support any recommendations that would benefit the region's economic prosperity in future by speeding up the city deal's delivery."
Deputy economy minister Lee Waters welcomed the report and said it gave the Welsh Government, local partners and the private sector "confidence to invest in the region and bring about economic growth and change".
A joint statement with the UK government added: "Our commitment to the Swansea Bay City Deal is strong and both governments are determined to see it deliver for the communities of south-west Wales. Working with our regional partners, this review report will help us to achieve this ambition."