At least 6,700 Rohingya were killed in the month after violence broke out in Myanmar in August, Medecins Sans Frontieres (MSF) says.
Based on surveys of refugees in Bangladesh, the number is much higher than Myanmar's official figure of 400.
MSF said it was "the clearest indication yet of the widespread violence" by Myanmar authorities.
More than 647,000 Rohingya have fled into Bangladesh since August, MSF says.
The aid group's survey found that at least 9,000 Rohingya died in Myanmar, also known as Burma, between 25 August and 24 September.
"In the most conservative estimations" at least 6,700 of those deaths have been caused by violence, including at least 730 children under the age of five, according to MSF.
Previously, the armed forces stated that around 400 people had been killed, most of them described as Muslim terrorists.
A case for the International Criminal Court?
Jonathan Head, South East Asia correspondent
There have been plenty of detailed reports by journalists and researchers, based on interviews conducted with refugees, which make it hard to dispute that terrible human rights abuses took place at the hands of the security forces.
But many of these reports focussed on the worst cases; there are several media reports about a massacre at one village called Tula Toli. Some Rohingya I interviewed told me they had fled in fear of violence, but had not actually experienced it.
This well-researched figure by MSF suggests the operation conducted by the military was brutal enough to raise the possibility of taking a case to the International Criminal Court (ICC) for crimes against humanity.
The problem would be that Myanmar has not ratified the Rome Statute of the ICC and is not bound to co-operate with it. Bringing a case would require the approval of all five permanent members of the UN Security Council, and China has until now given its full support to the Myanmar government's handling of the crisis.
The military crackdown began on 25 August after Rohingya Arsa militants attacked more than 30 police posts.
After an internal investigation, the Myanmar army in November exonerated itself of any blame regarding the crisis.
It denied killing any civilians, burning their villages, raping women and girls, and stealing possessions.
The mostly Muslim minority are denied citizenship by Myanmar, where they are seen as immigrants from Bangladesh. The government does not use the term Rohingya but calls them Bengali Muslims.
The government's assertions contradicted evidence seen by BBC correspondents. The United Nations human rights chief has said it seems like "a textbook example of ethnic cleansing".
"What we uncovered was staggering, both in terms of the numbers of people who reported a family member died as a result of violence, and the horrific ways in which they said they were killed or severely injured," MSF Medical Director Sidney Wong said.
According to MSF:
- 69% of the violence-related deaths were caused by gunshots
- 9% were due to being burnt to death in their houses
- 5% were beaten to death.
Among the dead children below the age of five, MSF says more than 59% were reportedly shot, 15% burnt to death, 7% beaten to death and 2% killed by landmine blasts.
"The numbers of deaths are likely to be an underestimation as we have not surveyed all refugee settlements in Bangladesh and because the surveys don't account for the families who never made it out of Myanmar," Mr Wong said.
In November, Bangladesh signed a deal with Myanmar to return hundreds of thousands of the refugees.
MSF said the agreement was "premature" pointing out that "currently people are still fleeing" and reports of violence have come even in recent weeks.
The group also warned there was still very limited access for aid groups into Rakhine state.
The Rohingya are a stateless Muslim minority who have long experienced persecution in Myanmar.