This is interesting, since from teardowns it looks like there's literally just a AAA battery in there:
Why not just solder the contact? If it's soldered, you should be able to swap the battery with a soldering iron. Sure, heating the iron and cleaning it will take much more time than swapping a battery in a spring-loaded housing, but I've spent just as long opening battery compartments held shut with tiny stripped screws.
The better ones use a double pulse for a cleaner weld without the need for any flux.