I've been trying to wean myself off phones completely since the demise of the SE, but life in Moscow without a phone is basically impossible (taxis, deliveries, electric scooters, bikes etc. all become pretty much unavailable).
As a compromise I've been using the Palm Phone[0] for a while now (before that the Unihertz Atom, which is quite the chonker, albeit with a small screen). It's the size of a credit card and has terrible battery life, but it can do all the things I need so I've just learned how to deal with it.
I don't really have a solution for music on the go yet, as you can't connect normal headphones to modern phones without adapters (and those adapters actually draw a surprising amount of power if they have a DAC!) and Bluetooth is completely useless in large and densely populated cities. Maybe some iPod-like devices are still around ...