West Ham have conceded no less than eight goals in their first two matches, and Hermansen has been at the centre of the criticism, with experts claiming that he is not up for the job after his lucrative transfer from Leicester City.
"Confidence is low. The goalkeeper doesn't make it any easier - he's having a really difficult time. He probably needs to be changed", said Sky Sports expert Jamie Redknapp after the humiliation.
"He's already made high-profile mistakes. Areola might have to come in. He doesn't fill me with any confidence right now. If you look at the goals they've conceded, he's been at fault for quite a few of them. It was the same situation at Leicester last year."
"I'm not saying he can't be a good goalkeeper and he can't be a success. But sometimes you have to look at the situation and actually do someone a favour."
The Guardian also blamed Hermansen for being at fault at Chelsea's fourth goal, saying: "Mads Hermansen flapped at a corner and gifted a fourth goal to Caicedo. West Ham cannot defend their box. The next corner brought more confusion, Hermansen bullied before Trevoh Chalobah scored the fifth."
West Ham signed Hermansen for around £20 million in a five-year deal with a one-year option, after a fee was agreed following several weeks of talks between the clubs.
West Ham are at the bottom of the Premier League and face a difficult away trip to Nottingham Forest on Sunday, August 31st.