Nearly half of youngsters cared for in England's children's homes are in places far from their home and family.
The education secretary has criticised the "indefensible" practice of sending troubled children to "far-flung" children's homes, many in tough areas.
New data shows there are 16 councils which place all their children outside their own local authority area.
In three areas; Rochdale, Brighton and Shropshire, 75% of those in children's homes are from other areas.
A Department for Education report, Children's Homes Data Pack, reveals that some 4,890 children are accommodated in children's homes.
Of these, 46% of are living in homes out of their local authority area, and nearly a third, 30%, are living more than 20 miles from their homes.
It is thought this could make them more vulnerable to potential abusers, as they are away from from their family, friends and social networks.
Whereas 58% of children in foster care are placed with families within their local council area.
Mr Gove said the practice of "decanting" children in this way was "indefensible".
"So, too, is the fact that more than half of children's homes are in areas with above-average crime levels," he added.
But local council leaders said there were very good reasons for placing children away from their homes.
David Simmonds, chairman of children and young people's board of the Local Government Association, said: "The flexibility to place children away from the area where abuse or neglect has brought them into the care system can be a vital way of giving them a new beginning away from these problems.
"There are very good reasons why some children in residential homes are placed outside their home area. This could be for their own safety, to break gang affiliation, to place them near other family members or to access specialist services.
"The historic problem of the clustering of lots of children's homes in a small number of areas that may suffer from higher deprivation is an area that we agree needs to be addressed. That is why councils, with their knowledge and understanding of local areas, should have greater say over where private care homes are set up.
"Residential children's homes play an important role in caring for some of the most vulnerable children in society at difficult times in their lives and councils have a key role in making this happen."
The report was commissioned following two separate inquiries into the children who go missing from care and face exploitation by gangs. It reveals publicly the locality of children's homes, many of which are now run by private and voluntary providers rather than local authorities themselves.
It was prompted by an abuse case in which nine men from Rochdale and Oldham, Greater Manchester, were jailed in 2012 for running a child sexual exploitation ring. The men, who exploited girls as young as 13, were given sentences of between four and 19 years.
Mr Gove also hit out at what he described as "bewildering" secrecy rules surrounding children, in an article in the Daily Telegraph.
He described a situation where it was very difficult to gather basic information about care homes and the children in them, and he said this could have hindered the police and helped individuals and groups seeking to harm children.
The education secretary said his department was unable to gather details about where children's homes were located and who was responsible for them, while the regulator Ofsted was not allowed to pass information to the police because of regulations, he added.
But others say there was simply a lack of clarity about what information could be shared between agencies, and that local authorities are able to give details of names and addresses of children's homes both to the police and the DfE.
The Information Commissioner has made it clear there was nothing to stop local authorities and Ofsted from sharing information on care homes with the DfE.
Barnardo's UK director of strategy, Puja Darbari, said "out of sight must never be out of mind when we are talking about vulnerable children in residential care".
She added: "It is scandalous that criminal gangs know the whereabouts of these young people while those who are meant to protect them do not.
"Barnardo's have long been calling for more accountability and justification when children are placed out of area."
Ellen Broome, policy director at the Children's Society, said many parts of the care system were not working.
"Children are being placed miles from home into run-down areas, the quality of some care homes is unacceptable, and children who run away from care are not kept safe.
"Children in care are extremely vulnerable. Most will have already suffered serious abuse or neglect at home. And people that want to exploit children are deliberately targeting children in care and children's homes."