The government is updating guidance on adoption in England to stress that consideration of a child's race should not be a barrier to finding a home.
Social workers have long been urged to seek child-parent racial matches.
But Children's Minister Tim Loughton said there was no reason to stop white couples adopting black or Asian babies.
Children from ethnic minorities are over-represented among those seeking adoption, but it typically takes three times as long to place them.
Mr Loughton said that in some cases social workers placed "too great an emphasis on finding the 'perfect match'".
"We know that a child tends to do better if adopted by a family who share their ethnic and cultural heritage," he said.
"Although the law and guidance is clear that due consideration needs to be given to language, religion, culture and ethnicity, this isn't translating into practice. It is much better that a child is adopted by loving parents than left waiting for their future to be decided," he added.
The current guidelines already say that race should be taken into consideration but should not be a block to an adoption.
But officials said Mr Loughton wanted to stress this point, and to encourage a shift in the way social workers operated.
Mr Loughton said it was "unacceptable" that the number of children placed for adoption had dropped by 15% this year.
He urged local authorities to make more use of voluntary sector adoption agencies - who he said had about 200 families on their books willing to take "difficult-to-place" children.
Official figures show that 2,300 children were placed for adoption last year, compared with 2,500 the previous year, and down from 3,400 in 2005.
In about 20% of cases identified as suitable for adoption, no placement is found.
Hugh Thornbery, director of children's services at the charity Action for Children, welcomed the government's focus on adoption and on finding suitable placements for children.
The organisation runs its own adoption agency, and he said that its experience showed it was possible to recruit more black and ethnic minority adopters with targeted outreach.
Most of the children being adopted from care were not babies, he said.
"They will have been in their community with their birth parents for some while before they come into care, they will have learned various things about their language, religion and culture and just to cut all that off can be really damaging," he said.
He said, however, there was a view that some adoption agencies had become "dogmatic" in seeking matches.
As society had become more diverse, so too had children's backgrounds - with many having heritage from two or more different ethnic backgrounds.
"It's about trying to find as best as possible a family that can give that child the best possible future," he said.
Research into transracial adoptions showed that the majority worked well, and it was not clear whether those that broke down did so because of racial factors, he added.
BBC News website readers have been sharing their thoughts. Here is a selection of their comments.