One option is to use 3.7v lipos[0] with a regulator[1], which can be done pretty cheaply:
[0] https://www.aliexpress.com/item/10pcs-3-7V-260mAh-Lipo-Batte...
[1] https://www.aliexpress.com/item/5pcs-Ultra-mini-DC-3-7V-4-5V...
[0] https://www.aliexpress.com/item/10pcs-3-7V-260mAh-Lipo-Batte...
[1] https://www.aliexpress.com/item/5pcs-Ultra-mini-DC-3-7V-4-5V...