Miscarried and aborted foetuses were kept in a hospital mortuary for up to four years, it has emerged.
Manor Hospital, in Walsall, said the foetuses were not released for cremation because staff failed to send the correct forms to the mortuary.
The hospital's chief executive, Richard Kirby, said he was "sincerely sorry for distress caused".
Walsall Healthcare NHS Trust blamed an administrative error and said it had since improved its procedures.
Mr Kirby said the mistake had come to light following a Freedom of Information request which asked how many foetuses had been retained for more than a month.
"We are sincerely sorry for any distress or concern we may have caused," he said.
"We have had a problem with our internal processes, which has meant that in a small number of cases, disposal of remains has not happened when it should have done.
"Those remains have been kept safely and securely in the mortuary at the hospital."
Review into failure
He added the hospital was contacting families directly to apologise.
The vast majority of the samples, 73, were below twelve weeks in gestation.
In total, 52 of the embryos were recent, dating from 2013, but there were two dating back to 2010.
Walsall Healthcare said it disposed of around 700 foetal remains each year.
The hospital now faces an inspection by the Human Tissue Authority on 25 March to ensure its procedures are up to scratch.
Mr Kirby said an external management review into the failure was also being carried out. However, he added, nobody had been disciplined.
A dedicated information line has been set up on 0345 8357626.