Maternity services at a hospital have been downgraded to "inadequate" after inspectors found a "potential risk for baby abduction".
The Care Quality Commission (CQC) inspected Bedford Hospital's maternity services in November following "14 whistle-blower inquiries".
Inspectors observed staff, women and visitors leaving maternity wards "without challenge".
A hospital spokesman said it had taken "immediate action" to improve.
The unannounced inspection was carried out after staff raised concerns about the safety of the service which "could lead to risk of harm to patients", the CQC said.
Maternity services at the hospital were previously rated as "requires improvement" after an inspection in August 2018.
The rating was downgraded to "inadequate" due to "concerns about staffing levels, insufficient training for staff and a poor culture amongst employees", the CQC's chief inspector of hospitals said.
The report found there was "a risk women and babies would be exposed to harm as there was not a formal system in place to ensure they followed the appropriate pathway of care".
Inspectors raised concerns with the hospital about "potential risk for baby abduction" on the labour ward and maternity ward.
While visitors had to be let into the wards through a staff-operated "secure door buzzer", inspectors found "anyone could exit" and observed people leaving without being challenged.
"In addition, we asked three members of staff for the trust's baby abduction policy, but they could not locate it," the report said.
It added some staff were "unaware" of any baby abduction drills or scenario training which meant they "may not know what to do in such circumstances".
The CQC warned Bedfordshire Hospitals NHS Foundation Trust that it must make "significant and immediate improvements" and served it with a "warning notice" in relation to its maternity services.
Although rated "inadequate", the report highlighted that infection risk was "well controlled", medicines were "well managed" and the premises and equipment were "visibly clean".
It also said staff "knew how to protect women from abuse and the service used monitoring results to improve safety".
David Carter, chief executive at Bedfordshire Hospitals said: "We took immediate action in the areas identified by the CQC and will continue to make improvements.
"As an organisation committed to delivering excellent care, the CQC feedback was disappointing for us to hear but we are focussing our effort on ensuring that the Bedford maternity service is able to meet the standards of care we aspire to."
Shane Hall, from the union Unison, said: "We've long raised concerns about the risks of low staffing levels in Bedford Hospital's maternity unit for NHS workers and expectant families alike.
"It still shouldn't have taken this kind of damning intervention from the CQC for the trust to address problems at the maternity unit but we're glad the hospital is now working with us to improve things for local residents."