Multisensory perception The look and the sound of food plays a more important part in our perception of flavour than you might have thought. Professor Charles Spence, contributor to Heston's show, explains all.

In this article

'Superadditive' personalities The stronger sense of eating

Hearing things The sound of the sea

We taste with all of our senses. While it probably comes as little surprise that smell and taste play a critical role in determining the flavour of the food that we eat, you might be surprised at just how much of our flavour perception is actually determined by the appearance of - and even the sound of - the food and drink itself.

Tactile cues also play a big role in our appreciation of foodstuffs: just think how unappealing food tastes when it is accidentally served at the wrong temperature (such as food served cold, when it should be hot or vice versa). To see how temperature cues can contribute to our perception of flavour, try this simple experiment:

Psychologists have recently started to quantify the relative importance of each of the senses in terms of their contribution to determining our overall multisensory perception of food flavour.

It seems that the human brain actually combines the information from each of our senses according to a number of very specific rules. So, for example, our brains tend to combine weak signals (such as the combination of a very weak taste with a very faint odour) in a 'superadditive' way that gives rise to a perception of flavour that's far more intense than you would expect simply by combining each of the sensory impressions when they're presented individually. You can experience this for yourself by trying this experiment:

Sensory cues

Watch out though, because if you get the combination of sensory cues wrong (by combining the taste of strawberry with the 'incongruent' smell of a savoury chicken soup, say) then the brain will not be impressed and it will give a subadditive response: that is, a response that is far lower than would have been elicited by either of the sensory triggers had they been presented in isolation.

Interestingly, it seems that the particular sensory cues that your brain will integrate in a superadditive manner (i.e., that it perceives as being congruent) will depend upon where you grew up. So, for example, those people who have grown up in the UK, and who have frequently experienced the combination of a sweet taste together with the smell of almond in their diet (as found, for instance, in Bakewell tarts), will tend to integrate the smell of almond and the sweet taste of sugar in a superadditive manner.

Brain science is beginning to help explain why it is that what tastes so pleasant to the people from one country can taste so bad to someone who has been brought up in another country

By contrast, the brain of someone who has grown up in Japan will not integrate sugar and almond in a superadditive manner (since they will not have come across that particular combination of taste and smell in Japanese cuisine). Instead, Japanese people show a superadditive response to the smell of almond when it has been paired with a salty taste (since that combination of smell and taste is common in Japanese cuisine, especially in things like pickled condiments). Thus, brain science is beginning to help explain why it is that what tastes so pleasant to the people from one country can taste so bad to someone who has been brought up in another country.

Another important rule that psychologists and brain-scientists have discovered about how the human brain combines the signals reaching each of the senses is known as 'sensory dominance'. That is, our brains use the most accurate of our senses when trying to figure out what is out there in the world around us, and this 'cognitive short-cut' even applies when we are trying to decide what exactly it is that we are eating and/or drinking.

Our eyes are generally very accurate in terms of their ability to tell us what something is, and hence our perceptions tend to be dominated by what our brains see, rather than by what they smell or taste or feel, etc. One wonderful recent example of the visual dominance over flavour perception comes from researchers in Bordeaux, France, who were able to show that people (even trained wine tasters) can be fooled into thinking that they are drinking a glass of red wine simply by colouring white wine red using an odourless food dye. The people (more than 50 people enrolled on a university wine course) perceived the white wine as having the bouquet of a red wine when it was coloured red.

Crucially the wine tasters did not perceive any of the odour notes in the coloured wine that they had previously reported when drinking the untainted white wine. Details of how to try this demonstration of sensory dominance on your friends at home are included in this experiment:

It's important to note that our perception of food is also determined by the sound it makes as we eat or drink it. A couple of years ago, Heston and I were able to show that people's perception of the freshness and crispiness of potato crisps could be systematically changed simply by altering the sounds that people heard when they bit into the crisps.

In particular, whenever we boosted the high frequency components of the crisp-biting sounds (we placed a microphone by peoples' mouths and the crisp biting sounds that were picked up by this microphone were fed through a graphic equalizer before being fed back to the participants over a pair of headphones), people would judge the crisps as being both fresher and crisper.

By contrast, when we attenuated the very same frequency components (that is, those above 2 kHz), our participants didn't like the crisps as much because they thought that they tasted older and staler. In fact, all of the crisps tasted exactly the same.

We have also shown that people's perception of the carbonation of soft drinks is determined to a certain extent by the sounds of the bubbles popping in the glass or what people hear when they're drinking. Thus, although we don't often pay much attention to the subtle sounds that we make whenever we eat and drink, our brains are continuously monitoring the signals going into our ears, and using them in order to help determine what exactly it is we are eating or drinking and just how much we like it.

In our latest research, Heston and I have been able to show that people's perception of the flavour of food can also be influenced by the sounds (or music) that is played in the background environment while they are eating. So, for example, in one recent experiment, we gave people two scoops of bacon and egg ice cream, one after the other. They had to rate the relative strength of the bacon flavour versus the egg flavour in each scoop. To our surprise we found that when the people heard the sound of farmyard chickens squawking, then the ice cream tasted much 'eggier' to them, whereas when we played sizzling frying bacon sounds over the loudspeaker system, people really thought that the ice cream had a much more prominent bacon taste.

In fact, both scoops of ice cream came from the same batch. This experiment therefore shows that our brains not only monitor the sounds of the foods that we're eating, but also process the sounds in the environment in which we are eating, and integrate them into our overall food perception.

The incorporation of the multisensory cues in the environment into our perception of the food and drink that we are consuming might then help to explain why so many of us have had the experience of buying a cheap bottle of great-tasting wine on holiday in the Mediterranean, only to find that when we get it home and open it in front of our friends that it suddenly tastes awful. What happened? Well, while on holiday your brain was taking in all those pleasant environmental cues such as the smell of the salty sea air, the warmth of the sun on your skin and the sound of the waves crashing on the beach and none of those environmental cues are present when you try the wine back in your own living room, hence it no longer tastes as good.

Heston has recently incorporated this insight into his menu at his restaurant in his 'sounds of the sea' seafood dish: diners are given a personal stereo playing the sounds of waves crashing on the beach to listen to while they are eating the seafood dish. Our research has shown that getting those environmental cues right, like recapturing the sound and smell of the sea, can help to make the dish more enjoyable. Try listening to the sound of the sea, with your favourite seafood dish.

Discuss the results of your experiments, and any food combinations you've tried, with fellow foodies on The Food message board.