Slaven Bilic says West Ham are "weak" and "have a problem" after the 4-2 defeat by West Brom, their fourth Premier League loss of the season.
The Hammers were 4-0 down after an hour, the second successive game in which they have conceded four goals.
It is their worst start to a Premier league season since they were relegated in 2010-11, despite spending over £50m in the transfer window.
"We have a problem we have to solve and that starts with me," said Bilic.
"I am the manager and have to do everything to turn this around and solve that problem."
West Ham are now 17th in the table with four defeats in five games, having conceded 13 goals.
They have also lost to Chelsea and Manchester City in the league, with a 1-0 win against Bournemouth their only victory so far.
Against Watford last week West Ham were 2-0 up before losing 4-2.
The Hammers enjoyed over 70% possession at West Brom but gifted the Baggies four goals through poor defending, and Bilic said he was "embarrassed" at half-time.
The Croat, who led West Ham to seventh last season in his first campaign in charge, added: "The problem is big.
"We can't say it is unlucky with some situations, or improve individual mistakes and we are going to be all right. It happens so many times since the start of the season - too many times.
"If we continue to defend like this as a team we can't win a Premier League game.
"We are weak and we have a problem. I don't want us to react, I want us to act. It is no time to panic but we have to act.
"We need to be aware of the situation and I don't want to sound pessimistic, but I am and I have to live with this 24/7."
West Ham, who moved into their new London Stadium home at the beginning of the season, were knocked out of the Europa League at the qualifying stages for the second successive season.
In 2010-11 the Hammers lost their first four matches and drew their fifth, ending the season bottom of the Premier League.
'Like watching an under-10s team defend'
Former England and Newcastle striker Alan Shearer said West Ham's defence was "a horror show".
Speaking on Match of the Day, he said: "I know they have injuries and two new full-backs but this is not a problem that has happened this season. It started at the end of last season. They have conceded at least two goals in 12 of their last 15 Premier League games.
"It was a horror show, so bad it was like watching an under-10s team. It was laughable and extraordinary.
"I have not seen defending like that for a long time. It was so bad it was unbelievable."
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.