As a young man James's girlfriends were flavoured of rhubarb and melted wine gums.
And his schoolmates had a strong essence of sliced potatoes and strawberry jam.
He chose his companions not based on their personality or looks, but because of how their names tickled his taste buds.
James Wannerton has synaesthesia - a condition in which the senses mix together so that sensations we normally consider separate start to intermingle.
Studies from the University of Edinburgh suggest 4% of the UK population could be affected by this blurring of the senses.
Researchers there and at the University of Sussex believe this phenomenon could be advantageous, particularly when it comes to remembering things.
Dr Julia Simner, at Edinburgh University, and Professor Jamie Ward, at the University of Sussex, are looking at whether savants - people with extraordinary abilities - use synaesthesia to help accomplish their mind-boggling feats.
One well-known savant, Daniel Tammet, set the European record for reciting the greatest number of decimal places of pi in 2004.
He has written much about how, for him, numbers have associated colours.
"We believe one of the keys to savants' extraordinary abilities is synaesthesia, giving them advantages in their memory recall," says Dr Simner.
In James's case, his synaesthesia certainly helped him at school.
As now, every time he heard a sound as a young boy, he had an immediate and involuntary taste on his tongue.
Hearing the name Anne Boleyn, gave him a strong taste of pear drops, making some history lessons a treat.
In fact most monarchs in British history came with a specific taste, which meant he could reel them off with ease.
His word-taste associations also help him remember every turn on the way to work.
"My route is like a set menu. It begins with a jam sandwich and it ends with a Rolo sweet," he says.
And he can't imagine what it would be like to attempt to recall things without their associated taste, though he is quick to say he wouldn't place his memory in the realm of the savants.
He was once offered the chance to temporarily disable these sensory experiences , but chose not to take it.
"It would be like someone asking you if you wanted to switch off your sight for 20 minutes," he says.
As president of the UK Synaesthesia Association he meets many people with these types of abilities.
Synaesthesia can come in many forms as the different senses mix and melt together.
The most common type is grapheme-colour synaesthesia where people perceive words, letters and numbers as distinct colours.
A more recently described type is mirror-touch synaesthesia - in which tactile sensations are experienced when a synaesthete sees others being touched.
And Mr Wannerton's relatively rare form is known as word-taste or lexical gustatory synaesthesia.
There are many theories about why this condition exists.
Some people believe it is due to the cross-wiring of certain areas in the brain usually unconnected.
Others suggest everyone is born with these cross-connections and in most people these links melt away, but in synaesthetes they remain strong.
Dr Simner says: "We now know for a fact the brains of people with synaesthesia are different to other people in two ways.
"Brains of synaesthetes have extra clusters of connectivity and there are differences in the grey matter of the brain - an extra thickness is seen in certain areas."
Prof Jamie Ward has studied Mr Wannerton's brain in detail using brain scanning techniques.
When we've asked James to think about words which have good and less palatable tastes to him, we see many parts of the brain lighting up, including areas associated with taste, emotional processing and mental imagery, he says.
But there is still a long way to go in understanding synaesthesia, particularly in identifying genes responsible for the condition, researchers say.
The tendency to have synaesthesia runs in families, though relatives will often have different types of it.
And research as early as 1963 suggests synaesthesia can be induced by drugs such as LSD - opening up the possibility the condition may be in part due to an imbalance of chemicals in the brain.
There have even been some cases where synaesthesia occurs after an injury to the brain.
But most synaesthetes, like James, feel they have always had it.
As a child it really didn't bother him.
"It felt as natural as breathing," he says.
But there are times in his adult life that the constant bombardment on his palate can be a nuisance.
These days if he has to work with someone with an overpowering first name, he chooses to refer to them by their middle names, or just re-christens them with an new one.
And talking to people in a crowded room can taste a bit like putting lots of strange things into a food processor at once, he says.
And some people think it is all in his head, he says.
Prof Ward says: "It is in the head. It is how it got into the head that is the question. Is it rooted in the head in terms of how the brain is structured and wired?"
But most of the time James does not mention his synaesthesia.
"So what if I taste sounds or the women in the supermarket tastes like an apricot?
"It's not that that interests me. It's where it's leading to.
"The study of synaesthesia is leading to a far greater understanding of how each individual sees the world around them and we each do see it in very different ways."