Bright youngsters are being turned away from investment banks because of their lack of "polish", a report has said.
Candidates who wear brown shoes, "loud" ties or ill-fitting suits can fall foul of "opaque" codes of conduct, the Social Mobility Commission found.
Firms recruit for front office roles from a few elite universities and only hire those who "fit in", it said.
Chairman Alan Milburn said "arcane culture rules" were locking working-class candidates out of City jobs.
A spokesman for the British Bankers' Association (BBA) said: "The banking industry has made significant strides to improve social mobility at all levels but recognises that we cannot afford to be complacent on this crucial issue".
The report's authors found that managers placed as much importance on a person's speech, accent, dress and behaviour as their skills and qualifications.
"Relatively opaque codes of conduct also extend to dress," the report said.
"To provide one example - for men the wearing of brown shoes with a business suit is generally, though not always, considered unacceptable by and for British bankers within the investment banking (corporate finance) division."
'Can't wear that tie'
The report quoted one candidate, from a "non-privileged background", describing the feedback he received from a mentor after an interview.
"He said you're clearly quite sharp, but... you're not quite the fit for [this bank]... you're not polished enough... he looked at me and said, 'see that tie you're wearing? It's too loud. Like you can't wear that tie with the suit that you're wearing'.
"What kind of industry is this where I can be told that I'm a good candidate, I'm sharp, but I'm not polished enough?"
The report found that, when picking candidates for jobs that involved dealing with clients, managers often selected those who fitted the traditional image of an investment banker.
It quoted one candidate as saying that the typical corporate finance banker is "usually someone pretty smooth, very fast talking, well-connected typically, likely to come from a pretty decent background, likely to be very polished, probably come from a good family, and so on. And that would be stereotypical, but still true, I think".
The report noted that in the UK, 7% of children attend fee-paying schools, yet the Sutton Trust found in 2014 that 34% of new investment bankers had attended a fee-paying school.
Dress 'reassures clients'
Dr Louise Ashley, from Royal Holloway University of London, who led the research on investment banking, said: "Access to front-office roles in investment banking is extremely competitive for all candidates, but our research suggests students from less privileged backgrounds are less likely to get the top jobs - no matter how talented they are."
The report said that issues relating to dress may seem both superficial and relatively simple for individuals from all backgrounds to adopt.
"However, interviewees suggested that they do play a material role in the selection process, once again, as demonstration of 'fit'," it added.
Specific dress codes are considered by banks to "provide reassurance to clients about the quality of the service they will provide".
Mr Milburn, who served in Tony Blair's Labour cabinet, said: "Bright working-class kids are being systematically locked out of top jobs in investment banking because they may not attend a small handful of elite universities or understand arcane culture rules.
"While there are some banks doing excellent work in reducing these barriers, there are still too many that need to wake up and realise that it makes sound business sense to recruit people from all backgrounds."
He said it was "shocking" that some banks still judged candidates on their dress rather than their potential.
According to the BBA, a number of banks have specific programmes designed to widen access, while others - including Barclays, J P Morgan and Schroders - choose to work with organisations such as the Social Mobility Foundation (which is a charity and not the same as the Social Mobility Commission which is a non-departmental public body).
"We want to ensure that [the] best people can succeed in banking - regardless of their background," the BBA spokesman said.
The Social Mobility Commission study also concluded that less advantaged recruits faced barriers in breaking into the life sciences sector, where there is a tendency to recruit graduates from the most prestigious universities.
"Life sciences" employers covered by the report were those involved in pharmaceuticals, medical bio-technology and medical technology.
Joanne Moore, from ARC Network, who led the research on life sciences, said: "Both the investment banking and life science sectors need to do more to ensure fair and equal access to jobs.
"This is important, not just for our economy and society, but for individuals.
"In life sciences, graduate jobs often focus on a candidate's practical skill as well as academic criteria - which may be more limited for non-privileged students who are known to face barriers to taking up placements and internships."
The data on how many new employees in the life science sector attended fee-paying schools is unavailable, according to the report.
However, it pointed out that for professional-level scientific jobs, the profile "appears to be skewed to high achieving graduates from relatively privileged backgrounds".