Yeah, I moved from an SE to an 8+, then an XS Max. I love the new phones, but if they brought out a premium SE I'd swap to that in a second.
I kind of care about the design. It's elegant-- I like the flat camera, the chamfered edges and the brick-y form (which I appreciate they brought back with the new iPad Pro).
What I care most about though is the headphone jack. This is why I still keep mine around in spite of having another phone-- to listen to music.
EDIT: The fact that it's cheaper, so I could get more memory for the same price as a bigger model also contributes to it still sticking around primarily for music, but also with some games.