West Ham will pay £15m for a 99-year lease on the £429m venue on which conversion costs could reach £190m.
"I do welcome the fact that the future of the stadium has finally been secured," said Caborn.
"But we should also realise that the public sector is picking up the tab."
Premier League club West Ham will move from Upton Park to the Olympic Stadium from August 2016 and pay around £2m-a-year rent.
Hammers land Olympic Stadium tenancy
Under conversion plans, the roof will be extended and the seating capacity reduced from 80,000 to between 54,000 and 60,000, with a retractable system allowing the venue to be converted from a football stadium to an athletics arena within days.
Seats will slide over the running track to bring West Ham fans closer to the action.
After West Ham's contribution, the rest of the funds for the conversion will be drawn from a range of sources, including Mayor of London Boris Johnson's budget, a £40m loan from Newham Council and around £20m of borrowings by the London Legacy Development Corporation.
Caborn suggested the stadium could have been adapted at a fraction of the cost if it had been built with a post-Games switch to football in mind.
"This is the biggest mistake of the Olympics and lessons should be learned from this," he added.
"West Ham are basically getting a stadium costing more than £600m for just £15m and a small amount in annual rent.
"The mistake was made in 2006-7 when they [Olympic Board] ruled football out of a retro-fit design as was done successfully in Manchester with the Commonwealth Games stadium.
"I suggested retractable seating like the Stade de France in Paris but they insisted it should be a 25,000-seat athletics stadium.
"Time and again mistakes are made with Olympic Stadiums and the lessons should be learned for any future similar projects."
Johnson defended the public cost of a stadium that will also be used to stage community and commercial events.
"It's a deal they said could never be done," he said. "This fantastic stadium will not only host community sports, but rock concerts, athletics like the Diamond League this summer, all sorts of sports where the ball is not necessarily spherical.
"I think when you look at the deal, the income, which is going to come in from rent, hospitality and naming rights, will be very, very substantial.
"That means there will be no more subsidy from the taxpayer to keep the whole thing going."
Breakdown of conversion funding
Loan from Newham Council:
There will also be funding from Mayor of London Boris Johnson's budget
But Johnson's stance was opposed by Conservatives on the Greater London Authority, who called for the West Ham deal to be scrapped.
"The decision to give West Ham the Olympic stadium means Londoners and the nation's taxpayers have been landed with a poor deal," said their spokesman Andrew Boff.
"It is a disgrace that another £25m from the public purse will now be used to prop up this agreement.
"The deal should be scrapped immediately and a commercially viable alternative explored as a matter of urgency."
West Ham will now start a consultation process with fans over the new stadium.
Two of its stands will be named after former Hammers stars Trevor Brooking and Bobby Moore to mirror Upton Park.
West Ham vice-chairman Karren Brady said: "It was important to me that we struck a deal that would stand the test of time and that represented the right deal for West Ham United and our loyal and patient supporters.
"The consultation we have promised them will now begin and we can't wait to work with them to create a stunning new home that befits the pride, passion and tradition that the world associates with West Ham United."
This page is best viewed in an up-to-date web browser with style sheets (CSS) enabled. While you will be able to view the content of this page in your current browser, you will not be able to get the full visual experience. Please consider upgrading your browser software or enabling style sheets (CSS) if you are able to do so.