I also like for that device to be able to receive and make phone calls...
...so I just use an old Android device, to be more specific my old Motorola Defy from 2011. It runs Android (Cyanogenmod, the predecessor to LineageOS) 4.4, does the Wifi thing but also the phone call thing, runs DSub connected to my own Airsonic server. It also takes photos, runs Telegram and when in dire need even a browser although it takes patience to wait for it chewing through a Javascript-burdened "modern" site. I have a 32GB microSD card in the thing which is enough given its connectivity options. The battery, also from 2011, lasts for 2 to 3 days of intense use.
Small Android devices running Google-free AOSP-derived distributions are a good match for the role of "standalone" music players with the added benefit of allowing connectivity. Total costs for the latter are around €10/year with occasional use (Sweden, "pay as you go" SIM), total cost for the device is zero since I've used it for 12 years.