Di Canio has brought success on the pitch - with Swindon promoted as champions last season and now sitting third in League One - but their financial situation has come under scrutiny in recent weeks after Black's decision to sell up.
It is thought around £9m is owed to investors - including Fitton - and the club have considered entering administration in a bid to wipe the unsecured debt.
And Fitton says the main reason for him leaving the board was because of their financial make-up.
"When the club started to look forward there were two models to follow; there's my model where we try and balance the books. But there's another model clubs have followed, both successfully and unsuccessfully, of investing heavily to get where they want to get to.
"The problem with that model is that it's very expensive and doesn't always work. We've seen countless times how clubs have destroyed people by being too expensive.
"I favour the first model because its sustainable and has a chance of long-term survival. The second model is fine as long as you've got benefactors willing to fund it.
"I believe Andrew Black was willing to fund a push for the Championship over a period of three years, hence buying in to the Paolo Di Canio project.
"I was happy to have Paolo but I knew it was going to be expensive. I stepped down because I didn't want to be the person in the boardroom always saying we need to save money."
After the revelation of the club's money problems, current chairman Sir William Patey said they had several interested buyers and was quick to play down fears of administration.
And Fitton believes the club should not be considering it as an option.
"Administration works when clubs have massive debts that they need to leave behind but have an underlying sound business," he said. "The club's only debts are to the shareholders - Andrew Black, myself, Sir Martin Arbib and Jeremy Wray.
"We have the power to do something about that. Equally that involves having a conversation and trying to work out a deal with a potential investor. I think if someone came in you would find me at the front of the queue, wanting to have a negotiation to try and work out what's good for the football club.
"But the real problem is how to fund the club going forward.
"The current major shareholder doesn't want to go any further, and that's his choice, and I think it's a shame because he's so close to his objective but he appears, for whatever reason, to want out of it."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.