A lack of support from employers is preventing many men from taking paternity leave, a report has found.
A survey of employees and managers found that a quarter of new fathers took no paternity leave at all.
Fewer than one in 10 took more than their two weeks statutory leave.
The research by the Institute of Leadership & Management (ILM) blamed "ingrained" attitudes among employers, and concerns among new fathers that they could not afford to take leave.
Last year the government announced plans to allow parents to share their maternity and paternity leave - changes are scheduled to come in April 2015.
But the ILM said its research suggested the changes would have little impact if attitudes of employers were not addressed as well.
"The introduction of shared parental leave is a crucial step towards enabling more women to progress into senior roles," said Charles Elvin, ILM chief executive.
"Yet our research revealed cultural barriers are impeding the uptake of both two weeks statutory paternity leave and additional paternity leave."
He said there remained a "cultural expectation" within organisations that women rather than men will be the ones taking extended periods away from the workplace.
The report also said low levels of paternity pay discouraged new fathers, with just 9% surveyed receiving more than two weeks on full pay.
Responding to the report, Frances O'Grady, secretary general of the TUC, the trade union body, said pay was the major concern.
"Many dads simply can't afford to take time off, particularly as employers rarely top up their statutory pay."
Currently, employed fathers are entitled to either one or two weeks' paid paternity leave, though additional leave is also given if the mother returns to work and is not claiming statutory maternity pay.
Government plans involve extending this flexibility, allowing parents of newborns to exchange and share their leave.
"We want the introduction of shared parental leave to drive a real cultural shift and help working dads play a greater role in their child's early months," a government spokesperson said.
"Employers too can gain from a system which allows them to keep talented women in the workforce and have more motivated and productive staff."