Stagecoach has said it refused to accept pensions risks that could have been "well in excess of £1bn" for three rail franchises it was disqualified from bidding for in April.
Under one scenario the company estimated it could have been left with liabilities of £1.6bn.
Protections offered by the Department for Transport (DfT) were inadequate, the transport group said.
Rail firms are facing an estimated £7.5bn pensions gap.
Last month, the DfT rejected Stagecoach bids for the East Midlands franchise, the South Eastern franchise with Alstom, and a renewal of the West Coast franchise with Virgin Trains and France's SNCF.
The department disallowed the bids because they did not meet pensions rules.
On Thursday, Stagecoach said rail franchisees were being asked to accept more risk to plug the pensions gap, and that there had been "requests from the Pensions Regulator for substantial and not yet fully quantified increases in contributions" to the Railways Pensions Scheme.
It said previously there had been an understanding that the Department for Transport would stand behind any liabilities as each rail franchise ends. Rail franchises typically last seven to eight years.
However, Stagecoach said there was an "absence of any contractual protection" for rail franchisees over unknown pensions liabilities.
"As yet, there has been no final determination by the [Railways Pensions Scheme] trustees or the Pensions Regulator of the extent of the liability that would be borne by franchisees.
"While ultimately the Department for Transport provided limited protection against the risk in the specification for the three current franchise competitions, our assessment was it still left the successful operators with substantial risk which could not be assessed."
It said the Department for Transport had not ruled out rail firms making raised contributions from 2022, or from having to contribute more if employees refused any rise in employee contributions.
The Pensions Regulator said: "We are working closely with the scheme trustees, Rail Delivery Group and the Department for Transport to ensure the best possible outcome for pension scheme members.
"Part of this work is to ensure the scheme is adequately funded so that members receive the benefits they expect."