Dame Tessa Jowell says Premier League chief Richard Scudamore has undermined his support for women's football.
Sexist comments in private emails to friends were given to the Sunday Mirror by his former personal assistant.
"His championing of women's football, and women's football coming of age, is undermined by this," Jowell told BBC's Sportsweek.
Scudamore, 54, apologised for the "inappropriate" comments, calling them "an error of judgement".
Jowell, a former culture secretary, said the comments were "incredibly disappointing" and "unacceptable",
She added: "You can't have one position publicly and then be laughing privately."
In his statement, Scudamore said the comments were made in "private emails exchanged between colleagues and friends of many years."
"They were received from and sent to my private and confidential email address, which a temporary employee who was with the organisation for only a matter of weeks, should not have accessed and was under no instruction to do so.
"Nonetheless, I accept the contents are inappropriate and apologise for any offence caused, particularly to this person."
Jowell, who was culture secretary from 2001-07 and Olympics minister between 2007 and 2010, said: "In the world of social media and email, there is no public and private.
"What Richard needs to reflect on, and I think he's a decent man, is the disconnect between what he thinks privately and what he has tried to achieve publicly.
"I hope his public commitment to women's sport will prevail. He's shown contrition and will be mortified by this, as indeed he should be."
Asked whether he should consider his position, she replied: "That's a matter for his bosses.
"I expect and hope that, as a changed person, he will reflect and make absolutely clear that these unacceptable views he circulated privately, he will never express again."
Women in Football, a network of women which aims to improve women's representation at all levels of football, said they were "shocked" by the reports.
Of the 661 who answered, 436 said that they had witnessed sexism within the football workplace, while 382 said they had experienced sexism in the football workplace.
A spokesperson said: "We expect the Premier League to conduct a full and proper investigation. Sexism, as with racism and homophobia, is not acceptable in the workplace."
Ruth Holdaway, chief executive of the Women's Sport and Fitness Foundation, said: "Sexism of any kind is deplorable.
"To see someone in the upper echelons of sport thinking it is okay to refer to women in this way, in private or otherwise, is utterly appalling, let alone someone who has publically supported the case for equality.
"To have someone in such a senior role talking about women in this way is unbelievably disrespectful and, ultimately, hugely damaging."
Peter Kenyon, former Manchester United and Chelsea chief executive, told BBC Radio 5 live's Sportsweek: "Having known Richard both professionally and as a friend for 20 years, I find it most uncharacteristic.
"I think he's been the advocate of moving football forward in a whole raft of ways, not least in terms of supporting the women's game, which is very good at the moment."
Mark Palios, former chief executive of the Football Association, said: "The important thing is Richard has apologised and recognised it was unfortunate.
"I don't think it changes the good work he does on a day-to-day basis."