Hollywood has the particular problem that the executives would really like a formula to follow to make people like the films, the screenwriters would really like a formula to follow to write something that they can sell, and so on. There's a lot of commercial pressure for a factory to mass-produce plot and stamp it out into films. People liked a film? Make the exact same thing and see if they'll buy it again. Franchises help, because at least there's some incentive to shuffle around some different characters and plot elements.

Though the never-ending soap-opera of comics aren't really that great a fit for wrapping everything up in a three-act structure. (I'm still confused by why the Marvel films felt the need to kill off 90% of their villains in the same movie they debuted.) But the hero's journey is an attempt to answer to "how can we make a film as popular as Star Wars?" So we just follow that pattern, I guess.

Not that there weren't other patterns--the Disney animators independently arrived at their own storytelling rules, for example. Disney Animation: The Illusion of Life (by Frank Thomas and Ollie Johnston ) has a discussion of what they found worked for them and what didn't. It's worth comparing what they say there with the actual films and reflect on how the rules bore out in practice, of course. But it's another attempt at codifying a formula for appealing stories.

There's lots of attempts to try to describe the universal appealing story pattern. Whether narrative actually works that way has become bifurcated into separate questions: "what kind of stories are effective for humans?" and "what kind of story can we produce reliably as a commercial success?" are subtly different but have become conflated.