It is a "scandal" that some children start school unable to speak in full sentences, Education Secretary Damian Hinds has said in a speech.
And if children reach Year 1 unable to read simple words, they rarely catch up, "the gap just widens", he said.
He has pledged to halve the number of pupils starting school behind in early talking and reading skills by 2028.
A group of companies and charities have been brought together to work out how best to support families in England.
Educational researchers have long said that social mobility - or the lack of it - starts at home with what's known as the home learning environment.
The idea is that a home with a lot of books and other early learning materials, plus engaged parents giving their children quality time, talking with them and teaching them how to make letter sounds, for example, provides a good start.
But not all parents feel able to offer this kind of home environment or realise the importance of it.
'Not lecturing parents'
Researchers from the Education Policy Institute last week said the development gap between England's poorest pupils, and the others, was already at 4.3 months in the early years.
And that it grew to 9.4 months by the end of primary school.
Mr Hinds said, in the speech in London, that this early gap had a "huge impact on social mobility".
"The truth is the vast majority of these children's time is at home.
"Yes the home learning environment can be, understandably, the last taboo in education policy - but we can't afford to ignore it when it comes social mobility.
"I don't have interest in lecturing parents here... I know it's parents who bring up their children, who love them. who invest in them in so many ways, who want the best for their children.
"But that doesn't mean extra support and advice can't be helpful."
The Department for Education says 28% of children in England do not have the required language skills by the end of Reception.
But this rises to much higher percentages in deprived areas.
Mr Hinds also says he is particularly keen to use technology to build awareness of what parents can do to boost early language development.
The speech comes a few months after communication charities, and the Royal College of Speech and Language Therapists, published their own report on the lack of progress on improving speech and language services over the past decade.
It was based on responses to a major independent cross-government review of such services chaired by John Bercow, which was published 10 years ago.
Bercow 10 Years On said that communication was crucial to children's life chances but awareness of its importance among the public and decision-makers was not sufficient.
Services were inaccessible and inequitable, it said, and too often support for children's speech, language and communication needs was based on the available resources, rather than what was needed, leading to great variations across the country.
It also said some measures shown to have worked well had been cut to save money.
The government is yet to respond to that report.
Bob Reitemeier, chief executive of I Can, the children's communication charity, welcomed the plans but said the coalition of companies and charities must include experts with experience of supporting children with speech, language and communication needs.
"Research shows that in some areas of deprivation 50% of children are starting school well behind their peers in language development.
"Without support, the gap between these children and other children will continue to widen, year on year, putting their life chances at risk."