Drivers using a hands-free phone get just as distracted as those holding it in their hand, researchers have found.
Scientists at the University of Sussex found conversations can cause the driver to visually imagine what they are talking about.
This uses a part of the brain normally used to watch the road, the University of Sussex study said.
The findings made the case for all phones to be banned from cars, according to the lead researcher.
It is illegal in the UK to ride a motorcycle or drive using hand-held phones or similar devices.
Drivers can get an automatic fixed penalty notice if caught using one. They will get three penalty points on their licence and a fine of £100.
The law currently says drivers can use hands-free phones, sat navs and two-way radios, but if the police think the driver is distracted and not in control of the vehicle, they could get penalised.
Checking for hazards
The study involved 20 male and 40 female volunteers who took part in video tests while sitting in a car seat behind a steering wheel.
One group of volunteers were allowed to "drive" undistracted while another two heard a male voice from a loudspeaker 3ft (0.9m) away.
Those who were distracted by the voice engaging them in conversation took just under a second longer to respond to events, such as a pedestrian stepping off the pavement, an oncoming car on the wrong side of the road or an unexpected vehicle parked at a junction.
The study showed that asking a simple question - such as, "where did you leave the blue file?"- during phone conversations could mean a driver concentrates on an area four times smaller than normal, because their brain is imagining the room where they left the file, instead of checking for hazards in front of them.
Alice Husband's seven-year-old son Seth died in December 2014, two weeks after he was hit by a car driven by a woman who was talking to a friend on her mobile phone using the loudspeaker function.
Mrs Husband told BBC Radio 5 live she agreed with the coroner, who said at her son's inquest that the use of the mobile phone would have had an impact.
"If my son perhaps wasn't so excited and didn't run, if he walked across the road - he's a child - if the driver hadn't been speaking on her mobile phone, all of these things could have made a difference to him, but at the end of the day it was a terrible accident and nothing is going to bring Seth back," she said.
"I think even just using a CD player; obviously we all know it is distracting because you are looking away from the road and I think we need to be more aware of how any distraction influences our driving and how important it is not to be distracted by such things if you get a phone call."
Dr Graham Hole, senior lecturer in psychology at the University of Sussex, said the research laid bare the "popular misconception that using a mobile phone while driving is safe as long as the driver uses a hands-free phone".
"The problem is enforceability - it's very difficult for the police to tell if someone's using a hands-free phone," he said.
"But on balance, I think the law should be changed to get the right message across and make it absolutely clear that any use of a mobile phone while driving is hazardous."
Other studies have suggested that phone conversations in a car are more off-putting than listening to the radio or talking to a passenger, Dr Hole added.
A passenger chatting in a car is less distracting, the researchers argue, because both stop talking when the driver needs to concentrate.
Alice Bailey, from road safety charity Brake, said: "These are life and death decisions, these extra three car lengths is the difference between a child dying and a child living and we just think the law needs to change.
"We need one clear law. All phones, hand-held and hands-free, need to be banned in cars - the only safe phone is one that is switched off.
"How important is any phone conversation that lives are lost?"
Kevin Clinton, from the Royal Society for the Prevention of Accidents, said he was not surprised by the study's findings and also called for a law banning the use of hands-free phones in cars.
"Sadly, people continue to lose their lives on our roads in crashes caused by drivers who are distracted because they use a mobile phone," he said.
"This can so easily be avoided by all drivers switching off their phones while driving, and only checking messages once they have stopped in a safe place."