Some work done by the probation service in Gwent has been labelled "troubling" in an inspectorate's report.
HM Inspectorate of Probation said higher-risk individuals were "managed well" but the picture was "more troubling" for those of medium and lower risk.
"Too many people get too little meaningful attention from probation staff", the report said.
The Wales Probation Trust has been asked to comment.
It was the first such inspection in Wales since the UK government introduced its Transforming Rehabilitation Programme.
It looked at the quality of probation work carried out with criminals by two organisations under the umbrella of the trust - National Probation Services Wales (NPS) and the Community Rehabilitation Company (CRC).
While the NPS was working with higher risk individuals more effectively, the CRC's work with medium and lower-risk offenders was called into question.
"Overall, the work of the CRC in Gwent was troubling," the report said.
The CRC, which is run by Working Links, had an "impressive" community hub but its ways of working were "still in a state of flux", with staff morale "low", it added.
Inspectors found one in four people identified as low-risk had only one telephone call every six weeks as supervision.
The report said one in three of these should also have contact with unpaid work supervisors or other intervention staff if those arrangements were to work as intended.
"Inspectors concluded that this means too many people get too little meaningful attention from probation staff," the report said.
"Without meaningful contact, individuals are unlikely to develop the will to change their attitudes and behaviour."
Chief Inspector of Probation, Dame Glenys Stacey, added: "Assessing the risk that someone might pose is not an exact science and risks change over time.
"But, in our view, someone's circumstances can't be kept under proper review through a telephone call every six weeks.
"Some other aspects of the CRC's work are not operating as they should and it is taking a long time for things to bed down.
"Staff morale is low and sickness absence high, although we did find committed responsible officers working hard to support people and to help them to change."
She said the CRC was "performing relatively well", but this was sometimes "at a cost to other work that should be done".
Dame Glenys called on NPS Wales to ensure the quality of its work did not vary from place to place.
The report made a series of recommendations, including calls for improved access to rehabilitation and for the CRC to introduce measures to monitor how it was working and to keep its workload and staff numbers under review.