The Bishop of Lincoln, Christopher Lowson, has been suspended.
The Archbishop of Canterbury acted saying it was in relation to a safeguarding children inquiry.
Justin Welby added that "if proven" he would consider the bishop "would present a significant risk of harm by not adequately safeguarding children and vulnerable people".
Bishop Lowson said he was "bewildered" by the suspension and would co-operate fully.
In a statement, the archbishop continued: "I would like to make it absolutely clear that there has been no allegation that Bishop Christopher has committed abuse of a child or vulnerable adult."
Bishop Lowson said: "I am bewildered by the suspension and will fully co-operate in this matter.
"For the sake of the diocese and the wider Church I would like this to be investigated as quickly as possible to bring the matter to a swift conclusion."
The Bishop of Grimsby, David Court, will take on the leadership of the diocese.
"It should be noted that suspension is a neutral act and nothing further can be said at this stage while matters are investigated," the archbishop added.
As part of Operation Redstone, Lincolnshire Police has been investigating historical sex abuse cases.
A spokesperson for the force said: "The investigation is continuing into wider safeguarding issues and management decisions within the diocese."
'Turned blind eye'
In April, a BBC Panorama investigation found clergy and staff from the diocese were referred to police in 2015 over allegations church leaders "turned a blind eye" to claims of child abuse.
Lincolnshire Police and the Lincoln Diocese investigated 25 people over alleged abuse from a list of 53 names passed to officers, with three cases leading to convictions.
The programme claimed some of the names could have been referred years earlier as part of the Church of England's national Past Cases Review, which examined tens of thousands of Church records in 2008 and 2009 to discover whether abuse cases had slipped through the net.
Paul Handley, editor of the Church Times, said the suspension had "cast a cloud" over the bishop and his colleagues.
However, he said: "I think what people need to take from this is the Church is taking safeguarding seriously, and nobody is beyond reproach if something is discovered that needs investigation."
The Diocese of Lincoln covers Lincolnshire, North Lincolnshire and North-East Lincolnshire, making it the largest diocese in England by area.