There was a "significant risk" of bodies being mixed up at a mortuary, inspectors have found.
Areas where dead babies and adults were stored were not secure at Torbay Hospital, in Devon, the report said.
The hospital almost lost its licence to store bodies and carry out post-mortems following the the Human Tissue Authority (HTA) inspection in May.
Health chiefs said inspectors had visited again on Tuesday and found "all urgent actions" had been taken.
In May inspectors found that staff were not updating paper and electronic records for bodies that had been moved to a different fridge - with just a whiteboard updated.
A fridge for foetuses and babies in the busy sluice room of the hospital's labour department was said to have inadequate security.
Concerns were raised about rusty equipment and issues with drainage on one of the post-mortem tables, which posed "significant risks to the dignity of the deceased and health and safety of staff".
There was also a risk of visitors gaining unauthorised access to the mortuary, which has space for 38 bodies, and of "inadvertent and unauthorised" viewing of post-mortem examinations.
The report found that a temporary storage unit for 12 bodies for use at busy times had been in near constant use for at least a year.
Analysis: Jenny Walrond, health correspondent, BBC South West
The findings of the Human Tissue Authority are worrying in themselves and will be distressing for people who have lost a loved one at Torbay Hospital.
But perhaps more concerning is that we learn from the hospital's board papers that the trust's executive team are concerned that financial pressures on the NHS may be contributing to these sort of problems: "An increasing tolerance to risk may be exacerbated by the financial situation and reduced availability of capital for maintenance and equipment."
They are worried that greater risks are being taken, because there isn't the money to resolve problems. While they do go on to express a need for an increased focus on mitigating such risks, there is no doubt that the trust is struggling with the costs of maintaining and replacing buildings and equipment.
Sarah Bedwell, director of regulation at the HTA, said: "During our inspection we identified a number of issues, some of which we judged to pose a significant risk to the dignity of the deceased.
"We visited again earlier this week to assess the trust's progress and to ensure that it is on track to meet the agreed action plan."
Dr Rob Dyer, medical director for Torbay and South Devon NHS Foundation Trust, said the hospital had improved ventilation, installed new equipment, ensured good fridge temperature monitoring, and updated some policies and procedures.
"The HTA are satisfied that all urgent actions were taken and there is good progress in other areas," he said.