The train company contracted to run Transport for Wales services, KeolisAmey, has been fined £2.3m by the Welsh Government for poor performance.
Economy minister Ken Skates said the penalty notices were "vital", saying the company had not lived up to what was agreed.
Recent improvements must continue, he added. The company took over the £5bn franchise in October 2018.
KeolisAmey apologised for the disruption of the last few months.
It is not clear when the fines were imposed. Transport for Wales said it was a "overall sum" imposed over "different periods", with fines issued on a periodic basis.
The company was awarded the Wales and Borders franchise by the Welsh Government in 2018.
The day-to-day contract is managed by Transport for Wales - a part of Welsh Government which also provides the branding the train services use.
In December the organisation saw cancellations following the introduction of a new timetable. The company blamed staff shortages.
"So far, something in the region of £2.3m in penalty notices have been issued to KeolisAmey," Mr Skates told the assembly's economy committee.
He said the money would be reinvested in rail services.
Transport for Wales said the fines were triggered because the operator had not met agreements for the subsidy it gives it.
The cash has been paid by deducting it from the subsidy, TfW said.
"A key challenge has been due to a shortage of fleet available to operate on a day-to-day basis," a spokesman added.
Trains have been out of service due to modifications for people with reduced mobility, internal and external refurbishments and work to install wheel slip protection before the Autumn leaf fall.
Recently extra capacity has been provided for 6,500 more passengers.
Vikki Howells, Labour AM for Cynon Valley, told the committee she was dealing with constituents who faced disciplinary action in work because of their lateness, have lost wages and had to spend cash on bus fares because of recent disruption.
Simon Jones, Welsh Government director of economy infrastructure, told her a measure called "passenger time lost" had "driven" some of the penalty notices.
He said it focuses on the "busiest journeys" - creating more of a problem for the train company if there were problems at rush hour.
Mr Skates said the proportion of trains arriving within three minutes of the scheduled time had improved by 6% in the "latest period".
That has increased to 76.1%, but the minister said that was against a target of 77%.
He did not want to be issuing penalty notices. "However it is something I feel is absolutely vital in order to incentivise better performance and to ensure passengers know when performance is not what people expect money is being returned.
"Performance has not been what has been expected and has not lived up to what the contract stated and what was agreed."
He told the committee the contract with KeolisAmey includes a break clause.
"If failure was to be seen in the future of course we would have to respond accordingly by looking at the contract," he added.
Kevin Thomas, chief executive of KeolisAmey Wales, apologised for the disruption of the past few months.
"We recognise that we have not been able to deliver the quality of service that we aim to and that our passengers across the network deserve," he said.
"It is encouraging to report that we are starting to see some improvements, with a 24% reduction in network delay minutes following the introduction of our December timetable.
"We have welcomed the arrival of the Class 170's, the recruitment of over 200 additional train crew and the introduction of more than 186 new Sunday services."
Caerphilly Labour AM Hefin David welcomed the fines: "I have had assurances from Transport for Wales that things are set to get better through 2020 and I will be monitoring this as a committee member and in my constituency."