Ex-Liberian President Charles Taylor has asked to serve the rest of his war crimes sentence in Rwanda, claiming that being detained in the UK denies him the right to a family life.
Taylor's wife and children have been unable to visit him in County Durham, lawyer John Jones QC said.
He was convicted of aiding rebels who committed atrocities in Sierra Leone.
The UN-backed Special Court for Sierra Leone trial was held at The Hague on the agreement he was jailed elsewhere.
The overseas venue for the court case was chosen in case the trial sparked renewed unrest in West Africa.
An act of parliament was passed to allow for Taylor to serve his sentence in the UK, at the cost of the British government, following his conviction.
Taylor was sentenced in 2012 and arrived in the UK last October, having unsuccessfully challenged the decision to be detained there.
Legal papers have now been lodged with the court claiming that serving his sentence in the UK breaches his human rights.
The former leader, 66, says it would be more "humane" for him, and his wife and 15 children, if he was to return to Africa to complete his sentence.
His lawyer told BBC Radio 4's Today programme: "In the eight months that he has been in the UK he has not received a single visit from his wife and children.
"He has minor children. When he was in The Hague on trial, he had regular visits, they came, they saw him, they went back."
Mr Jones added: "What we are saying is the UK has a duty to ensure family life, not just for him but for his family. It's a clear duty under international law and English domestic law."
The UK would save money if Taylor was transferred, he claimed.
"He is not suing the British Government, he is not seeking damages from the UK and, on the contrary, for the UK taxpayer it would be much, much cheaper if he were to serve his sentence in Rwanda with all the other prisoners from the special court," said Mr Jones.
Mr Jones said visas had not been granted to members of his family as immigration officials were "not satisfied that they are going to return to Liberia after their visit to see him, which is ridiculous".
- 1989: Launches rebellion in Liberia
- 1991: Revolutionary United Front (RUF) rebellion starts in Sierra Leone
- 1997: Elected president after a 1995 peace deal
- 1999: Rebels take up arms against Taylor
- June 2003: Arrest warrant issued; two months later he steps down and goes into exile in Nigeria
- March 2006: Arrested after a failed escape bid and sent to Sierra Leone
- June 2007: His trial opens - hosted in The Hague for security reasons
- April 2012: Convicted of aiding and abetting the commission of war crimes - later sentenced to 50 years in jail
- Oct 2013: Arrives in the UK to serve the remainder of his sentence
He said Taylor was the only person convicted by the special court for SCSL to be transferred from their home continent and that he had the impression "there is a deliberate will to isolate him" - not just in the UK, but in the North of England.
Taylor has been kept in the hospital wing at HMP Frankland since his arrival as he is considered vulnerable, according to the document submitted to the court by his legal team.
An anonymous letter sent to Taylor, possibly from within the prison, threatening him with bodily harm and death demonstrates the "seriousness of the danger", the document adds.
Speaking to the BBC news website, Mr Jones stressed that Rwanda was a more humane place for Taylor to be, as "everything is different here" - from the "freezing cold" weather to British food.
Taylor was convicted on 11 charges including terrorism, rape, murder and the use of child soldiers by rebel groups in neighbouring Sierra Leone during the 1991-2002 civil war, in which some 50,000 people died.
The former Liberian leader was found to have supplied weapons to the Revolutionary United Front (RUF) rebels in exchange for so-called blood diamonds.
The rebels were notorious for hacking off the limbs of civilians to terrorise the population.
Taylor has always insisted he is innocent and that his only contact with the rebels was to urge them to stop fighting.
A Ministry of Justice spokesman said: "Charles Taylor is being treated in accordance with the United Kingdom's obligations and in the same way as any other prisoner in England and Wales."
The Foreign Office said it did not comment on individual prisoners or individual visa applications.