Banks have been warned that it's "not fair" to automatically blame customers for money lost through a scam.
Last year, customers transferred nearly £240m to fraudsters, only a quarter of which was refunded by financial firms.
The Financial Ombudsman service said fraudsters' growing sophistication meant it was wrong to assume losses were due to customer carelessness.
The service, which resolves customer disputes, said victims who lost money had a good chance of being reimbursed.
"It's not fair to automatically call a customer grossly negligent simply because they've fallen for a scam.
"That's especially true in light of the sophisticated way criminals exploit banks' security systems - and convince customers that their money is at risk," said Caroline Wayman, chief ombudsman and chief executive of the Financial Ombudsman Service.
She said when a customer had fallen for a fraud or a scam, typically both they and the bank denied they had done anything wrong.
Customers hit by fraud
One customer called Brian contacted the Ombudsman service after his bank refused to refund him £7,000 in a text message scam.
Brian received a message he thought was from his bank and unwittingly gave out his security details and passcodes.
As a result, the bank said he had been grossly negligent and refused to refund the money.
After reviewing the details, the Ombudsman decided it was a sophisticated fraud, and that the fraudsters had gained Brian's trust and therefore his actions were reasonable.
They forced the bank to reimburse Brian's £7,000.
Another fraud victim called Jas contacted the Ombudsman over several large withdrawals on her monthly statement which she didn't recognise.
The bank said she must have made them herself or given someone her pin number.
The Ombudsman found her account of what happened and the details about where she kept her card "consistent and plausible".
The bank was forced to refund the disputed transactions because the Ombudsman said the bank had also failed to show Jas had been grossly negligent.
Ms Wayman said there was "a very high bar" for banks to prove customers had been "grossly negligent", meaning the banks were not liable for the money their customer had lost.
"If they can't do that, it's likely we'll tell them to cover the money their customer has lost," she said.
Examples of the tactics used by fraudsters including creating fake websites which look identical to banks' online systems - or text messages which look like they're from someone's bank.