Banks "too often" blame customers for falling victim to criminals carrying out fraud, according to the Financial Ombudsman Service.
Figures showed a 36% rise in complaints about how banks dealt with fraud and scams this year compared to 2019.
Debbie Enever from the ombudsman told BBC Radio 4's Money Box "work still needs to be done" on a voluntary code most banks signed up to last year.
UK Finance, representing banks, said stopping fraud was a primary focus.
It added that the voluntary code was "not working as intended" and called for legislation to better protect customers.
This Contingent Reimbursement Model Code - or CRM - means that banks should refund victims, if they are not at fault, when they have been tricked by criminals into transferring money out of their own banks accounts.
Between April to September 2020, figures from the Financial Ombudsman Service showed there were 6,603 complaints about banks from victims of fraud, up from 4,840 between April and September in 2019.
Fraudsters pretend to be from the police, the government or the banks themselves and use intense psychological pressure, often against elderly and vulnerable victims.
Last year, £456m was stolen from more than 100,000 victims using this method, destroying victims' financial futures and devastating them and their families.
But whilst the Financial Ombudsman Service only upholds around a third of general complaints made about banks, when it comes to complaints about this type of fraud it backs consumers in three quarters of cases.
"Under the code that came in last year, the presumption was supposed to be that consumers would get their money back unless they were at fault in someway," Ms Enever said.
"We're just not seeing that the banks are following through on that at the moment.
"We are always keen to let people know that they can come and complain to us [about banks and building societies] and the email addresses and phone numbers are on our website."
Leslie runs a small company in London and had £13,200 stolen from him by criminals who hacked his email account in June 2019.
He should have been covered by this code but after investigating for five months his bank, Lloyds, told him it wouldn't refund the money.
Leslie complained to the ombudsman which, in June 2020, backed his complaint but six months on Lloyds still hadn't refunded the money.
Money Box started investigating Leslie's case and within two hours of Lloyds being contacted by our reporter, Leslie was emailed to say he would be getting his money back.
"I'm obviously delighted to hear the money is returning... but this system is clearly broken and there needs to be a change," he said.
"When a bank promotes they have entered a code [to refund victims] they are asking customers to trust them to follow that code.
"My own experience suggests Lloyds at least are not deserving that trust."
Lloyds told us keeping its customers' money safe is its priority and it had a great deal of sympathy for Leslie.
It said it had already agreed to refund Leslie's money before Money Box got involved, adding that the timing was just a coincidence.
What is the Financial Ombudsman Service?
The Financial Ombudsman Service settles disputes between consumers and businesses that provide financial services.
It is the final word in complaints and banks and building societies must abide by its decision.
The agency has a two-stage process: an "initial view" handed down by an investigator and a "final decision" which is enforceable by law.
In 2019/20 it upheld an average of 32% of complaints in general about banks.
That figure nearly doubled to 58% in relation to complaints about frauds and scams.
In cases of push payment fraud that number was even higher at 75%.
Katy Worobec of UK Finance, which represents the main banks, said: "Fraud is a crime which has a devastating emotional impact on victims and its proceeds fund serious criminal activities which damage our society."
She told Money Box the primary focus was stopping fraud in the first place and helping customers stay safe.
In relation to the CRM code, Ms Worobec said it was not working as intended, admitted a lack of consistency from banks and was working on making it more effective.
She said the industry was also calling for further action, including new legislation, "to drive more consistent outcomes for consumers and firms".
You can hear more on BBC Radio 4's Money Box programme by listening again here.