Phonetic IMEs aren't the only input methods available. There are a number of input methods based off of character components for Chinese and Japanese that are generally an order of magnitude faster (not accounting for predictive text). It requires about the same level of memorization as learning Vim. The problem with phonetic input is that it doesn't narrow down the search space enough for typing to be open loop. It's essentially the equivalent of typing in IPA and then selecting the equivalent English spelling. The problem is that without predictive text the search is too broad for any given input. Kanji actually has a set of about ~500 atomic components that all characters can decompose down to, so if you just map these in a lossy way to the US keyboard or just the numpad for some input methods, the search space is narrow enough that you can get close to open loop typing.

You can also decompose to about 10 strokes, whose order when written is set, so you can map from sequences of those strokes.

The problem with all these input methods IMO is that they're not deterministic and therefore not open loop because that just wasn't one of the design goals.

The additional benefit is that you can apply whatever rules your input method uses to symbols from math or other languages, which IMO is far simpler than using LaTeX syntax if you just want to type a ‰ or ⊥ sign. My input method can type Chinese, Japanese, Greek, and Russian as well as some mischellaneous symbols.

Entirely mechanical component based typewriters were being developed in the 50s but never came to fruition. By the time electronic typewriters came along it was decided that it was just cheaper to use the existing physical keyboards designed for Ascii or numpads on phones and work around it with software. This seems to have been a consistent pattern for most languages regardless of the number of symbols or whether they were phonetic. IMO I believe if they had just standardized on a more specialized keyboard layout in the 80s IMEs would not exist. IMO you can likely deterministically type Kanji in a open loop way if there were maybe 2 to 5 times the number of physical keys as the standard, or the same number of physical keys with 4 more shift keys. The advent of predictive text the need for it is basically nil for the majority of people in the same way the majority of people have no need for LaTeX or Vim.