Charges for unarranged bank overdrafts could be banned, under one option being considered by the Financial Conduct Authority (FCA).
It said the charges for those who go into the red without agreement can be high and complex.
Earlier this month, the UK's largest lender, Lloyds, said it was getting rid of unarranged overdraft fees altogether from November.
Barclays has already stopped all unauthorised lending.
However, other banks charge about £6 a day, or up to £90 a month.
"We believe there is a case to consider fundamental reform of unarranged overdrafts, and whether they should have a place in any modern banking market," the FCA said, in its review into the high-cost credit market.
"Maintaining the status quo is not an option," said FCA chief executive Andrew Bailey. Unarranged overdraft fees were often "significantly higher" than payday loans, he added.
However, the FCA made it clear that an outright ban on unarranged overdrafts was only one option being considered.
It could impose a cap on charges, or demand some affordability checks before a bank lends money on an unplanned basis.
A year ago the Competition and Markets Authority (CMA) decided against a cap on charges.
Half of all overdraft users go over their agreed borrowing limit, according to the CMA. In 2014 such customers spent £1.2bn in charges as a result.
The banking industry responded by saying that customers were usually warned if they were about to go overdrawn, usually via a text alert on on a mobile app.
"When used sustainably, consumer credit is important for economic growth, and lenders work hard to ensure the balance is right between helping customers to borrow while ensuring longer term affordability," said Eric Leenders, head of personal banking at UK Finance.
The FCA has also highlighted concerns about the rent-to-own market, typically used by consumers to buy fridges, freezers and televisions.
"We think that is a sizeable issue, because people are paying three or four times more than if they used cash," Mr Bailey told the BBC.
The FCA said that one option might be for housing associations to provide such goods instead.
Mr Bailey said there were also concerns about motor finance, a worry already highlighted by the Bank of England.
"We're looking at affordability tests and the transparency of terms," he said.
The FCA will publish an update on this work in the first quarter of 2018.
As part of its review into high-cost lending, the FCA also looked at how the cap on payday loans was working.
It said that the cap, first imposed in January 2015, had delivered "substantial benefits" to consumers.
Since then, no one has had to pay more than 0.8% a day of the amount borrowed. The maximum they pay is no more than twice the amount they borrowed.
The FCA said its review found that the cap meant 760,000 borrowers in this market were saving a total of £150m a year, that companies were now less likely to lend to customers who cannot afford to repay, and debt charities were seeing fewer people struggling with ballooning borrowing from payday loans.
Mr Bailey said the FCA would continue to focus its efforts on what else needed to be done in this area.