The Bishop of Oxford has insisted he did act when a priest told him he was raped repeatedly as a teenager.
The Right Reverend Dr Steven Croft, who became Bishop of Oxford in September, was told of the allegation in 2012 by the priest, who has since retired.
Dr Croft said he followed up the allegations and those concerned were "properly supported".
But the man said Dr Croft "did nothing" and "couldn't get off the phone quick enough" when he told him.
"I would have thought he would have wanted it sorted out before he was enthroned [in Oxford] but he has persistently not engaged," he said.
'Three years of hell'
The victim, now in his 40s and who wanted to remain anonymous, said he was sexually abused by a priest in Yorkshire in the 1980s.
He told Dr Croft, who was then Bishop of Sheffield, but said he was "angry" at the lack of action.
The man added: "The past three years, for me, have been hell because I live with it every day.
"I never think of anything else, and thought I was tough, and I thought I was a grown man and I could cope.
"It never leaves you and it brings back all sorts of memories."
Dr Croft said: "Clearly this person feels... he didn't receive the support he needed and I regret that very much.
"In the course of a long conversation about many things he dropped in a reference to having endured this [sexual abuse] as a teenager."
'Carrying this burden'
The bishop said he made an appointment to visit the man, but had to cancel after receiving a separate complaint against the same person.
He said he had arranged for someone else to provide pastoral care and kept a written record of the actions he took.
Dr Croft said "anybody carrying this burden from the past should be encouraged to come forward".
Under the Church of England's clergy disciplinary measures a complaint must be made within one year of the alleged misconduct.
An application for permission to make a complaint out of time was considered in August but was rejected by a High Court judge.
West Yorkshire Police confirmed there was an ongoing investigation into a historical serious sexual offence in the Bradford area in the 1980s.