It may be the public school and Cambridge education. Perhaps the accent does not sit well.
There may just be an underlying indifference to a perceived elitist sport.
Whatever the reason, few have bothered to scratch beneath the surface of one of Wales' greatest modern sportsmen.
Tom James stands out from the crowd on the banks of the Thames at Henley Royal Regatta.
He is not the imposing physical specimen you might expect from an Olympic gold medal winning rower.
Standing at 6ft 3ins with 13 stone of muscle spread around his body, it is easy for him to be overshadowed by the bigger built rowers watching the day's racing.
James is easy to pick out, though, as he is the only other person at the event wearing jeans.
That's the thing about James; he confounds the preconceived ideas one might have about British rowers.
"I don't think we're going to get into any of the bars or restaurants today…" he said, chuckling as we compare clothing choices.
Jeans, casual shirts and trainers. Not the outfit expected of an Olympian at Henley.
The 27-year-old has only been back in the sport for a matter of months having sat out the past two years.
The first year off post-Beijing Olympics was the Welshman's choice; the second forced upon him by a back injury.
I wonder out loud whether he now feels like a professional rower once more.
"It's funny you put it that way," he replied.
"Having had the hip operation to help fix the back problem, I was tentative getting back into the boat and to start with I didn't do high intensity training.
"It has been a gradual process but now I've done trials, World Cup races and my body can deal with full tilt action now. So, yes, I do now consider myself to be a proper rower again."
After impressing in the trials, James has been dropped straight back into the coxless four boat in which he won the gold in Beijing three years ago.
His return meant that despite two World Cup wins in 2010, Alex Partridge was shifted to the eight-man boat.
"I'm a pawn in the whole thing, we're all just commodities really!," he said, a smirk coming across his face as he contemplates the lack of control a rower has over his career path.
"Jurgen [Grobler, head coach of Great British Rowing] decides who goes into which boat and which combinations will bring the most medals.
"It's my job to excel as a rower and sit wherever he puts me."
A double Olympian, James is aware of the pressure involved in rowing at home games in the iconic boat made famous by the likes of Steve Redgrave and Matthew Pinsent.
His manner is relaxed; the weight of the world seems far from broad shoulders.
There is, however, a pragmatic realism about how the experience of London 2012 will differ from his previous.
"Athens and Beijing are one thing," he said.
"But now we've got 70 odd million people in the UK willing us to win. It's a whole new ball game.
"I think we'll all feel the intimacy of the whole nation watching in a very much more personal way.
"I could be quite overwhelming to be honest. But it's not something to be scared of, but something to be aware of… to be ready for."
There is a ready humour which emanates from James.
Unlike many athletes, he seems to possess a combination of renewed perspective and passion for sport, perhaps borne out of two years on the outskirts.
"If I'd have come straight back to rowing I'd have gone mad," he said.
"I'd been doing the sport since I was 14 and the pressure, both mental and physical, increases the higher up the sport you rise.
"I needed to have a year away to do dangerous sports like skiing and to just sit on the sofa and get fat."
Whatever excess weight was gained in 2009 it has long since been shed and the focus is back. As our chat draws to a close, the inevitable subject of a repeat victory at London 2012 surfaces.
"I've thought about that a lot," James adds as he narrows his eyes and scratches his stubbled chin.
"It's a good question which I'd been mulling over for a while before I decided whether to do the Olympics again.
"I'd be very disappointed if we don't win. Our aim is to win a gold medal. I think it has to be."