That was the problem. The game's lead, Yu Suzuki, was not primarily a game designer. He was known to push ahead games with revolutionary graphics, see Space Harrier and Virtua Fighter. But he was nowhere near as good at game design as Miyamoto or Naka.
I loved Shenmue for the immersive experience which was unchallenged at the time. It was the first of its kind, and for a long time no other open world game paid as much attention to detail. But its traditional gameplay mechanics were weak.