"Rent-to-own" shops that sell appliances and furniture for small weekly payments but with a high interest rate face a price cap.
However, the financial regulator will not rush to impose the same restrictions on bank overdrafts.
The Financial Conduct Authority (FCA) has spent nearly two years looking at the cost of high interest borrowing.
It has now outlined a package of plans for rent-to-own, doorstep lending and catalogue shopping.
High-cost credit is used by three million people in the UK.
Single-parents aged 18 to 34 are three times more likely to have a high-cost loan - such as a payday loan, doorstep loan or pawnbroking loan - than the national average.
"The proposals will benefit overdraft and high-cost credit users, rebalancing in the favour of the customer," said FCA chief executive Andrew Bailey.
Campaigners had called for a cap on the interest and charges faced by those using high-cost credit, including overdrafts.
They said that cap on the cost of payday loans, introduced in 2015, should be a template for the rest of the high-cost credit market.
About 400,000 people have outstanding debt with rent-to-own firms such as BrightHouse from which they buy household appliances, paying the money back over three years.
After interest, they can end up paying many multiples of the cost price.
The FCA said it had seen cases when people had ended up paying more than £1,500 for essentials like an electric cooker that could be bought on the high street for less than £300.
"The FCA believes the harm identified in this market is sufficient in principle to consider a cap on rent-to-own prices. It will now carry out the detailed assessment of the impact that a cap could have on the rent-on-own sector and how it might be structured," the regulator said.
Such a cap would not be in place before April 2019.
John Glen, Economic Secretary to the Treasury, said the measures would help the most vulnerable avoid being stung by "dodgy deals".
That includes people like Kenneth Murray, who says he had to buy a laptop from a rent-to-own firm business as he could not get credit from a high street electrical store.
"I had multiple debts that I was trying to juggle, and no stable source of income. I ended up taking out loans to pay loans," he said, although he has now managed to halt this cycle.
More than three million people have dipped into an unauthorised overdraft in the course of a year, exceeding their agreed limit.
In 2016, firms made an estimated £2.3bn in revenue from overdrafts, with 30% of this from unarranged overdrafts. The charges for those who go into the red without agreement can be high and complex.
The majority of unarranged overdraft charges were paid by only 1.5% of customers, who paid about £450 a year in fees and charges, the FCA said.
The regulator is proposing that banks offer more information to customers, including mobile alerts, about when and how they go into the red.
It is also considering a ban on fixed fees, which can lead to relatively high charges for a small unarranged overdraft.
Overdrafts in numbers
- Those aged 35 to 44 are most likely to have some form of overdraft
- A total of 10% of all 18 to 24-year-olds have exceeded their overdraft limit in the last 12 months
Source: FCA, Financial Lives, October 2017
Part of discussions will be a "potential backstop price cap for overdrafts", but it is not an immediate proposal. The FCA said that it needed to overcome potential legal issues that had been encountered in the past.
Gareth Shaw, of consumer association Which?, criticised the delay: "Last summer, the FCA expressed serious concerns about how unarranged overdrafts work, and now almost a year later it is still refusing to take action."
Gillian Guy, chief executive of Citizens Advice, said it was disappointing the regulator had failed to protect users of doorstep loans and unarranged overdrafts as well as buy-to-own customers.
"There are some strong potential remedies on the table for people struggling with overdrafts - the onus is now on the FCA to act quickly," she said.
But Eric Leenders, managing director of UK Finance, said big banks were making overdrafts more transparent.
"This builds on the unarranged overdraft alerts already available and supported by the regulator, which have already reduced charges to consumers by as much as 25%," he said.
Greg Stevens, chief executive of the Consumer Credit Trade Association, which represents 300 lenders, welcomed the FCA's "cautious approach" and said it was "taking its time to get the balance right".