That's not the fear. The fear is that the player will get stuck and simply stop playing because it's frustrating instead of fun.
The way that devs determine where the line is for "most people that play get frustrated and give up" and "most people figure it out and keep playing"
It's certainly possible to design a game where the intuition of the player matches a less contrived looking world, bit it's a lot harder. On the other hand, having a convention that climbable surfaces will be blazed yellow, and teaching it early in the game is comparatively easy.
There's also aspects of pacing and focus to consider. A game that ebbs and flows in it's intensity often feels better to play than one that's high intensity all the time. So if your game is mostly about combat, you'll want to break up big fights with lower energy experiences. But those lower energy experiences really aren't "the game" in a sense, they're a kind of filler that makes the high energy experiences feel more fun. It can't be completely boring, but it should be easier than the primary gameplay experience and also have lower stakes. That's a big reason why do many actiony games have blatantly obvious climbing and puzzles. It keeps the player lightly engaged while letting them catch their breath before the next set piece.
It's not the only way to build games by any means, but it is a generally effective, and consistently reliable, template.