If you don't need the "UPS" capability, you should be able to run them without a battery.
i replaced a swollen battery on a cheapo hotspot with an 18650, but i had to remove the protection chip from the old battery and wire that up with my battery holder. if i was more industrious/desperate i probably could have used the one in the 18650, but that was more fiddling than i wanted
The PinePhone boots without a battery but it's modem won't.