Around 40% of homeowners with mortgages could struggle to move because they would not qualify for a new loan.
The growing band of potential "mortgage prisoners" is thought to number as many as four million, BBC News has learned.
Stricter checks on mortgage applicants were brought in a year ago.
Many lenders are strictly applying checks despite "transitional provisions" allowing banks to show flexibility if existing customers want to move or remortgage.
As a result, some people are trapped in a mortgage deal or have to pay much more.
"They've pulled the rug from under me and I'm stuck," said Siobhan Moloughney, from Bath, who had a home and a steady job but has ended up back with her parents along with all her belongings.
Her recent move fell through after her lender refused to transfer her mortgage, even though it was supposed to be portable.
Her sale went through so she has been knocked off the property ladder, watching house prices rise in Bath.
"It's really unfair," she said. "I'd already proved that I could afford the mortgage and made sure the mortgage was portable.
"I trusted the mortgage company to look after me. It's just cut me loose."
She was refused the same level of mortgage after her lender ran affordability checks on her income and outgoings.
Yet she has a clean repayment record, her earnings have not changed and she does not need to borrow more.
"It is a crazy situation," said Ray Boulger, of mortgage brokers, John Charcol.
He has calculated that up to four million people, or 40% of those who have a residential mortgage, could be affected in one way or another.
"People who have a perfectly good track record on their current mortgage and don't want to borrow more money are finding they are being refused because of these new rules and the way they are being interpreted," he said.
Matthew Whittaker, from the Resolution Foundation, which has been monitoring the mortgage prisoner issue suggested that "between 35% and 40%" of mortgage holders could be hit.
The rules, imposed a year ago by the Financial Conduct Authority, require lenders to take a detailed look at bank statements.
Some have been scrutinising the cost of gym memberships, milk bills, childcare and pension contributions, creating a trap for home movers.
Also on the danger list are owners with interest-only mortgages, the self-employed and people who are deemed too old to borrow.
Chris Baytopp, 59, from Hertfordshire, lost out heavily over his move, because of his age.
"It doesn't make any sense," he said. "Affordability isn't an issue with me."
Many mortgage providers are questioning whether borrowers like him will be able to meet their payments once they start picking up their pensions.
His mortgage was another so-called portable one which his provider refused to transfer. It was a fixed rate, with a penalty fee for early redemption.
So, even though he was able to find a new lender when he and his wife moved to downsize, he had to pay fees of £4,000.
"I'm angry. We had plans for that money, now it's gone into the ether," he said.
The Council of Mortgage Lenders (CML) said stricter treatment could be justified, even though transitional provisions drawn up by the regulator, the FCA, allow providers to make exceptions as long as applicants are not increasing their borrowing.
Paul Smee, the CML's director general, said that a review of affordability could be in the customer's best interest.
"The rules are tougher for a reason and the industry has to work within them," he said.
"If there are areas where there are glitches, then or course we must look at ways around the glitches."
But Pat Bunton, chairman of the Association of Mortgage Intermediaries, said that captive customers were being treated unfairly and that "large swathes of prime customers are trapped".
He advised victims to consider taking their cases to the Financial Ombudsman Service.
The Ombudsman Service has already upheld some complaints over portable mortgages.
"We will need to see good reasons from the lender as to why they won't allow them to port their mortgage," a spokesman for the service said.
"We also have questions for lenders who decline to offer lending despite consumers financial circumstances not changing, or in many cases getting better or more consistent.
"If the risk hasn't changed - or has lessened - we may ask a lender to reconsider if they can't demonstrate good reasons for declining to port the mortgage."