The Archbishop of Canterbury said he was "embarrassed" and "irritated" that the Church of England invested indirectly in online lender Wonga.
It comes after the Most Reverend Justin Welby told Wonga the Church would try to force the firm out of business by helping credit unions compete with it.
But the CofE later admitted it invested in funds that provided money for Wonga.
Archbishop Welby told the BBC he wanted the Church's investment rules to be reviewed following the row.
Lambeth Palace said an independent inquiry would be launched into how "this serious inconsistency" occurred.
The amount of Church money indirectly invested in Wonga was about £75,000 out of investments totalling £5.5bn, according to the archbishop.
"It shouldn't happen, it's very embarrassing, but these things do happen and we have to find out why and make sure it doesn't happen again," Archbishop Welby told BBC Radio 4's Today programme.
He said Church investment managers "didn't pick up" that they had put funds in a "pooled investment vehicle" which, through its investments, had bought into Wonga.
The Church's Ethical Investment Advisory Group "recommends against investment" in companies which make more than 3% of their income from pornography, 10% from military products and services, or 25% from other industries such as gambling, alcohol and high interest rate lenders.
The Church also "reserves the right" not to invest in companies with "unacceptable" management practices, according to its website.
Archbishop Welby said the 25% level for firms which deal in high interest rate lending was "probably too high" and he would ask the advisory group to review it.
"I think we have to review these levels and make sure that we are consistent between what we're saying and what we're doing," he said.
But he said it was difficult to decide exactly which businesses were unethical, giving hypothetical examples of a clothes company which made socks for the military or a hotel which provided pornography through the TVs in its rooms.
He said the Church had to operate in the "real world", adding: "If you exclude any contact with anything that directly or indirectly at any point gets you anywhere bad, you can't do anything at all."
Gavin Oldham, of the Church's Ethical Investment Advisory Group, later said financial managers had been aware of the indirect Wonga investment and "more could have been done" with that information.
But he said the amount of money involved was a "tiny holding within that particular investment".
Payday firms offer short-term loans, often at high interest rates, and have been accused of leading people into more debt.
Archbishop Welby said he did not want to "drive legal payday lenders out of business" if that left people in deprived areas with no choice but to use "loan sharks".
This week, Archbishop Welby told Total Politics magazine he had met Wonga boss Errol Damelin and had "bluntly" told him "we're not in the business of trying to legislate you out of existence; we're trying to compete you out of existence".
Mr Damelin, in response, said he was "all for better consumer choice".
But, after the archbishop's comments were widely reported, Financial Times journalists looked into the Church's own investments and discovered links with Wonga.
The paper reported that the Church's pension fund, which claims to explicitly ban firms involved in payday lending, had invested in US venture capitalists Accel Partners - a company that led Wonga fundraising in 2009.
Archbishop Welby, a former oil industry financial executive who sits on the Parliamentary Commission on Banking Standards, has previously lobbied for a cap on high interest rates charged by loan companies.
BBC religious affairs correspondent Robert Pigott said the archbishop accepted that taking on payday lenders was a risky project which might not work.
But our correspondent said news that the Church had invested in funds that provided money for Wonga had "raised the stakes even further" and meant Archbishop Welby's plan "has to succeed".
London Mayor Boris Johnson - who was once criticised for accepting sponsorship of the Tube from Wonga - said there was "no doubt their rates are usurious".
"[The archbishop] is not turning over the tables of the money lenders, he's bringing in his own money lending tables," Mr Johnson added.
"It's a very interesting interpretation of the gospels."
Simon Hill, of anti-capitalist network Christianity Uncut, said: "What we do with our money is part of how we live out following Jesus, part of how we follow Jesus's example, of siding with the poor and actually put our money into things like renewable energy, social housing, things that, in itself, will help society."