I had one and love it. I’m able to use it without limitation one handed (often while carrying something or otherwise unable to use both hands), something I struggle with on my X. I also don’t like having things in my pockets so the small size was fantastic.

I was gifted an Apple Watch and that has helped with a lot of things I would do single handed on the SE previously, like quick reply text messages and getting directions.

There’s a market for small high end phones. It’s likely just not big enough for Apple (or anyone really) to care about.