I think the problem is you're making a horror game using cute anime girls. It just doesn't make sense to build a horror environment and then have a dog girl squeal at you or a girl with twin tail drills T-posing at you at the end of a hallway.
Maybe if it was a horror or macabre themed vtuber like
@SandyCat normally likes, but it just doesn't make sense with Hololive.
I don't know, I didn't mind it that much and I think back then with how shitty tech was seeing your game file crapping out was a real scare. Probably wouldn't work today, but it also kept enough variety in the mindfucks that it didn't get old.
They probably also took some inspiration from MGS where you had to swap the controller for the Psycho Mantis fight.