There need to be more male role models for children in early years education, the school leaders' union NAHT says, as only 3% of teachers in this field are men.
The union says it is important for all children to have positive male role models and to understand that men can be interested in education.
The Department for Education says a group has been set up to look in more depth at the number of men working in early years - and consider what more could be done.
But how can more men be encouraged to apply for the roles?
1. Challenge the stigma - it is not just for women
"I think it is a deep-seated culture issue," says ex-head teacher James Bowen, now a director for NAHT.
"Because of that, men don't see men doing it so they just don't consider early years as an option."
He suggests that there is a feeling early years teaching is the type of job only women do that prevents men from considering the roles.
"It is incredibly hard to change culture that has existed for decades," he says.
"The only way to get a culture change is you have to get people people talking more."
He believes that a government advertising campaign would help - and suggests one way to promote the job to men might be to talk to fathers who are dropping off their children at nursery.
Sally Bates, head teacher of Wadsworth Fields Primary School in Nottingham and a member of NAHT's early years sector council, says society can be suspicious of men working in the sector, resulting in low recruitment.
"This perception needs to be loudly challenged at every level," she says.
2. More money - salary and bursaries
Shaddai Tembo, from the Bristol Men in Early Years Network, is a family support worker at a children's centre in Bristol.
He says early years roles are still seen as jobs for women and work needs to be done to change perceptions.
"It is about getting the word out and changing the public image," Mr Tembo says.
But he believes the other issue is money.
"The first years of your life are the most important and people working there are still on minimum wage," he says.
Early years teachers are similar to qualified teacher status in schools, he says, "but pays nowhere near as much".
"The EYT role itself is great, but the pay disparity turns many people to teaching instead," he says.
"There is not the career path there should be for the early years."
He wants the job to be perceived as more of a profession "rather than an easy job to do".
Elaine Sagar, from the Sunflower Group, which manages a set of nurseries in Greater Manchester, agrees.
"There needs to be a lot more money brought into the sector to encourage men in," she says.
"It needs to come down to positive promotion. The government looked at lack of males or lack of teachers coming into the profession, so they put a lot of bursaries out there.
"I think it's important they push that down into early years to get those role models in."
3. Gender diversity targets and training
For Jeremy Davies, from the Fatherhood Institute think tank, it is not just about the salary.
He regards the early years sector as "institutionally sexist" and says the government should act on "several fronts" to address this.
Action should include gender diversity targets and publication of annual data on the gender composition of the workforce, he says.
He would like to see gender diversity training for early years managers, practitioners and commissioners - and for schools careers advisers, and a high-profile, well-funded recruitment campaign "targeting fathers and other men with experience of looking after children".
"Yes, of course, more pay would help - but that needs to be part of a bigger conversation about the professionalisation of the sector, which is not just about attracting more men," he says.
"Think about the kind of long-term investment put into transforming the teaching profession in recent decades, through programmes like Teach First.
"Given what we know about its supreme importance for children's future outcomes, why wouldn't you extend that approach to early years education too?"
Teach First is an initiative to recruit graduates to teach in deprived areas, which in 2013 became the biggest destination for university leavers, according to a survey of major UK employers.
4. Change the job name
Ms Sagar also suggests that changing the job title of some of the roles could help.
"Maybe we change the term from nursery nurses to early years professionals," she says.
"The stigma needs to be removed to encourage more men into the sector, and it's so important for those early years.
"We need these men to come in lower and start at the very beginning with the children."
She believes that the government also has to do more to promote roles in early years.
Patrick Foley, head teacher at Southborough Primary School in Kent, says: "Working in an early years setting is a rewarding profession and can be enormously enjoyable.
"Men should be encouraged to join these professions and the barriers to men making these choices should be investigated and removed.
"More men in these key roles would improve outcomes for children in their early years, which would have tremendous benefit for all children."