A postmaster says the Post Office spent £320,000 suing him over £25,000 he was falsely accused of stealing.
Lee Castleton, from Bridlington, East Yorkshire, was made bankrupt after he lost a two-year legal battle with the Post Office.
A recent ruling found problems with the Post Office's accountancy software could instead be to blame for the losses.
The Post Office has apologised and accepts its "past shortcomings".
For two decades, the Post Office pursued hundreds of its workers over accounting discrepancies with its Horizon IT system, accusing people of theft, fraud or false accounting. Many were fired, made bankrupt or even sent to prison.
Mr Castleton described how he and his family were abused, including an incident where his daughter was spat at, because "local people presumed I was a thief".
The former stockbroker, who bought a seafront Post Office branch in 2003, said he began to notice thousands of pounds in losses from his accounts within months of taking over the business.
Despite calling the Post Office's helpline nearly every day for three months, he said: "We just couldn't understand where the losses were coming from."
Following an audit, Mr Castleton's branch was found to have a £25,000 shortfall. He was subsequently suspended.
The Post Office spent the next two years pursuing him for the missing money through the civil courts.
"The Post Office decided to make an example of me," said Mr Castleton.
He was forced to defend himself in London's High Court because he had no money to hire a lawyer. When he lost the case, he was made bankrupt.
Mr Castleton is now one of more than 550 former postmasters who will get a share of a £58m settlement from the Post Office.
The agreement was reached in December last year, when the High Court ruled that technical problems with the Post Office's Horizon IT system could instead be to blame for losses - something Mr Castleton and the other claimants had always believed was the case.
'No space for honesty'
Internally, some Post Office staff had also grown concerned about what was happening.
Speaking out for the very first time, one former manager told the BBC's File on 4 the Post Office had "zero interest" when he questioned why increasing numbers of postmasters were being blamed for losses totalling millions of pounds.
The former staff member, who wishes to remain anonymous, said he voiced his concerns after reading internal documents. "I didn't like what I was finding but was told it would be better for my career to 'move on and let it go'," he said.
He said he and a number of colleagues "began to feel increasingly uncomfortable but there was no space for honesty, no desire for open dialogue".
"The people running our stores were being arrested going to prison, losing their life savings, having massive mental health issues. It felt as though doing the right thing no longer mattered, it was all about saving the image of the Post Office".
He said: "A fellow member of staff asked one manager why they thought there was such a high volume of fraud. The manager laughed and said 'because half the post offices are run by ex-police who took early retirement or former pub landlords so they're probably used to fiddling the books'.
"I think it was meant as a joke but it betrays a subtle contempt for the people on the front line."
"Protecting the brand"
Ron Warmington, from the forensic accountants 'Second Sight', was brought in by the Post Office in 2012 to look at around 150 cases.
In his first ever broadcast interview, he told File on 4 that as his findings became more critical of the Post Office, it became increasingly difficult for him to access the information he needed.
"It's the way that corporations behave when things go wrong," he said. "And it was in that area that we found it so strange, the way the Post Office was reacting, presumably to protect the brand."
In 2015, the Post Office ended the investigation without notice but not before Mr Warmington had spoken to dozens of postmasters.
"I've spent decades dealing with some of the worst criminals in the world," he said. "What struck me here was that I wasn't dealing with people like that.
"These were ordinary people that had simply found something that they couldn't deal with. They sought help but didn't get it and were put in a situation that frankly was life-changing for them."
The Post Office declined an interview but said it has "accepted its past shortcomings" and "sincerely apologised" to those affected.
File on 4's 'Second Class Citizens: The Post Office IT Scandal' is on BBC Radio 4 on Tuesday 11 February at 20:00 GMT and available afterwards on BBC Sounds.