New research challenges the view that people would migrate to other nations as a result of climate change.
The study by the International Institute for Environment and Development (IIED) said people tended to stay in their own country.
It added that people had been uprooted by a range of factors, not just changes to the climate.
The report challenges the widely held view that climate change would trigger an influx of cross border refugees.
It had been forecast that as many as 50 million environmental migrants might be on the move by 2010.
The IIED report says new studies in Bolivia, Senegal and Tanzania found no evidence that environmental degradation would result in large flows of international migrants.
Lead author Cecilia Tacoli said most displaced people wanted to stay as near home as possible, adding that most stayed within their own borders, although there was a serious lack of information within countries facing widespread internal migration about the numbers involved.
But Dr Tacoli added that if sea level rose as projected then it was likely that there would be many international environmental refugees from small island states.
"Environmental change undoubtedly increases the number of people mobile," she told BBC News.
"But catastrophe like droughts and floods tend to overlap with social and structural upheaval, like the closure of other sources of local employment that might have protected people against total dependence on the land," she told BBC News.
"Of course we need to act on climate change, and rich nations have a moral obligation to help poor people affected by it. But it's often easier and quicker to address the socio-economic factors."
The report observed that families living in areas of environmental decay would often choose to send one family member to a city to earn money to bolster rural incomes. This was a positive outcome of migration, Dr Tacoli suggested.
The projection that 50 million people would be environmental refugees by 2010 was made back in 1989 by Mustafa Tolba who, at the time, was executive director of the UN Environment Programme (Unep).
In 2001, a report by the International Federation of Red Cross and Red Crescent Societies said more people were forced to leave their homes because of environmental disasters than war. It estimated that approximately 25 million people could currently be classified as environmental refugees.
Dr Tacoli said the absence of data in countries facing most internal migration made it impossible to verify these figures.
A Unep spokesman told BBC News: "The scale and pace of environmental change, from climate to the loss of forests, freshwaters and other key ecosystems, is increasing the vulnerability of humanity. Many scientists are also now warning of 'tipping points' that could trigger irreversible changes to the planet's life-support systems.
He added that how these changes, if left unaddressed would displace people was the subject of a "great deal of academic debate".
"It is complex and it is unclear," he explained, "but what is clear is that such risks are rising, not receding.
"Whether they are termed environmental refugees or not, they way societies manage or mismanage their environmental assets are likely to increasingly define movements and migration patterns over the coming decades--what would you do if your town or village ran out of water or productive land? "
The idea that people displaced by shifts in the climate, especially from poor nations, could materialise as a migrant problem for rich polluting nations became a political weapon for scientists and environmentalists demanding carbon cuts.
Environmentalists would point out that internal migration also presents huge challenges to nations, and that the world has not begun to see the sort of potentially catastrophic changes envisaged under the more extreme scenarios of climate change.