Justin Welby has apologised for "hurt and pain" caused by the Anglican Church to the LGBT community.
The Archbishop of Canterbury made the comments as he defended the decision to restrict a liberal US branch of the Church for allowing same-sex marriage.
He said it was not for Anglican leaders to "divide the Church" and that union would be "painful as well as joyful".
But he added it was a "constant source of deep sadness that people are persecuted for their sexuality".
His comments followed an agreement between Anglican leaders on measures to curb the US Episcopal Church, including barring it from decision-making for three years.
The agreement - made at a four-day meeting of 39 Anglican primates in Canterbury - also upheld a "traditional doctrine" of marriage as being between a man and a woman.
Episcopal leaders and campaigners have criticised the decision, with people gathering to protest at Canterbury Cathedral.
The head of the Episcopal Church, Presiding Bishop Michael Curry, told BBC Radio 4's The World Tonight, the experience of the gay and lesbian community was similar to that of African-Americans, adding: "Even after emancipation there were those who were excluded and segregated."
Anglicans have been divided on the issue of homosexuality and same-sex marriage since the US Episcopal Church ordained an openly gay bishop in 2003.
Leaders said that stance was a "fundamental departure" from the faith of the majority in what is the world's third largest Christian denomination.
Speaking at the Primates 2016 conference, Archbishop Welby said facing protesters, particularly those from Africa, was a reminder of the "pain and suffering of many LGBTI (Lesbian, gay, bisexual, trans and intersex) people around the world".
He said: "For me it's a constant source of deep sadness, the number of people who are persecuted for their sexuality.
"I don't have the right to speak for everyone. I wanted to take this opportunity to say how sorry I am for the hurt and pain, in the past and present, the Church has caused."
But defending the decision to curb the US Episcopal Church, he said the "unity shown by the primates" was "going to be painful as well as joyful and remarkable".
"It is not for us to divide the body of Christ, it is not for us to divide the Church," he said.
The rift over the US Episcopal Church's stance on same-sex marriage and homosexuality dates back to the ordination of openly gay Canon Gene Robinson.
He was made a bishop of the Episcopal Church's New Hampshire diocese in 2003.
More than 100 senior Anglicans had used an open letter less than a week ago to urge the Church of England to repent for "discriminating" against lesbian and gay Christians.
However, the Anglican leaders in Canterbury said the Episcopal Church's approval of gay marriage was "a fundamental departure from the faith and teaching" of the majority of Anglicans.
The Anglican leader denied suggestions the US Church was being sanctioned but said it was facing "consequences" and said the decision was supported by the "overwhelming majority" of Anglican primates.
Asked if the Church's position made it look outdated, he admitted it would in some parts of the world but not others.
He said: "It makes us look out of line in the US and UK, yes, but not in many other parts of the world, no.
"We are a global Church and that means that there are different views in different places."
He also said the mistreatment of gay people in some countries, especially those where homosexuality is criminalised, remained a "major concern".
BBC religious affairs correspondent Caroline Wyatt
There was an atmosphere of significant mistrust at this meeting, perhaps hardly surprising given the animosity of the past years between the most traditionalist of the provinces and the most liberal.
The disagreement over Anglican doctrine regarding same sex marriage is probably a more fundamental one than the divisions that raged over female priests.
But the subtext of the meeting has been the power struggle just beneath the surface over who should lead the Anglican communion in the future - the countries of the global south, where congregations are growing, represented in part by Global Anglican Future (GAFCON) or Canterbury, its traditional leadership based in an increasingly secular global north where congregations have been in decline for many years.
Through diplomacy and negotiation - and a tight media blackout which succeeded until almost the last day - the Anglican Communion has pulled through as one, for now.
The careful wording of the resolution agreed by the majority of primates ensured there was no mention of the words "sanction" or "punishment", although the strength of feeling about the Episcopal Church's unilateral action was abundantly clear.
Nonetheless, even as GAFCON welcomed the deal, its leadership added that: "This action must not be seen as an end, but as a beginning."
So even though the Anglican Communion has not "moved into separate bedrooms", as some had expected, it is continuing to live in a deeply divided household.
Read more from Caroline Wyatt here
The Global Anglican Future Conference (Gafcon), which represents conservative Anglican leaders worldwide, said it was pleased by the outcome but said "this action must not be seen as an end, but as a beginning".
But Presiding Bishop Michael Curry told BBC Radio 4's World Tonight that the decision would bring "great pain" to gay and lesbian members of the Anglican Communion.
He said: "Those who are baptised followers of Jesus - whether they are straight gay, lesbian, bisexual or transgendered - if they're committed to following the way of Jesus, they should have equal access to all of the services and sacraments in the life of the Church."
He also defended the decision of the Episcopal Church to change the canon to allow for same-sex couples to be married in church.
"Obviously it's a serious disagreement. We believe the decision we've taken is the right one and they've expressed their displeasure and disagreement and we respect that."
Other gay Christians and supporters of equal marriage were critical of the decision.
Labour MP and former Anglican minister Chris Bryant, who is gay, said: "I've finally given up on Anglican church today after its love-empty decision on sexuality.
"One day it will seem wrong as supporting slavery."
Archbishop Welby is to appoint a task group to rebuild trust in the Anglican Communion.
Anglicans, whose roots are in the missionary work of the Church of England, are the third-largest grouping of Christians in the world, behind Roman Catholics and the Orthodox.