Normal people care about music theory as much as they care whether you use jemalloc vs tcmalloc. "Easy listening" is a much more useful everyday definition for them than whatever musicians may want it to be.
Normal people who don't care about music at all, sure they can call it "easy listening" and I won't bother arguing. For anyone who cares about music or even history at least a little, it's worth knowing that jazz is very important to the history of music.
It's possible to care only in some modes of your being.
Kind of Blue is far from “easy listening.”
Compared to Anthony Braxton, "Kind of Blue" is elevator music.