It seems to me that mathematical higher dimensional matrices are, in the broad scheme of programming languages, a corner case. Certainly a big one and an important one, but not one that was ever likely to drive general purpose languages.
Indeed, I'd argue that if that was your primary use case you are and always were better off with a relatively custom language (relative to programming as a whole), because the needs are so different and the wins so big using an environment set up to support those needs that you'll want that in the end. You can fuzz the line with things like NumPy, because all these lines are fuzzy, but dedicated support will be a big win in the end.