Linear algebra is a relatively straight forward subject. I won't say easy because there are bits that aren't, and I struggled with it when I was first exposed to it in college. But in graduate school revisited it and didn't have a problem.

So, I really agree that it's about timing and curriculum. For one, it appears somewhat abstract until you really understand geometrically what's happening.

So, I surmise that most non-mathematicians don't have quite the mathematical maturity to absorb the standard pedagogy when they first approach the subject.