To me it does sound exactly like that too in English.
I'd wager it's more obvious in the Japanese version because Japanese is the exceptional language. There are only 44 syllables in Japanese (English has about 16,000) and one would probably still notice this in otherwise unintelligible distorted speech.
Spanish is about as phonetically consistent as Japanese. Most languages have simpler phonology than English. It's not about the number of possible syllables, it's about the rules to go from text to phonemes. The rules in English are immensely complicated and inconsistent. Meanwhile, I can accurately describe Spanish phonology, such that you'd be able to pronounce ~any Spanish word (English loanwords excluded) accurately including stress, in about one page. Written Japanese lacks pitch accent information, but otherwise works similarly.
(By the way, your stats are off; modern Japanese has about ~106 possible syllables (mora) by rough count).
Most TTS software out there will be better at English than any other language despite the more complex phonology. Then it's solely about distinguishing language characteristics and I think the amount of syllables would have an effect in that. Japanese has a lower information density than English (at the same speed) so with the same amount of distortion Japanese should be more recognizable.