Since his comment mentioned that he does not care about weight and thickness then batter cases is a perfect solution for that, it's easier to purchase a battery case then an original replacement batttery
Also, another important aspect of a removable battery is that some people like it for privacy. If you can't remove your battery, you can't really assume your phone is truly "off". Even if you can, if you really care about that, you would wait a while to make sure any small internal batteries/transistors have used what power they could hold. Other than that, I'm not sure of a way to ensure the baseband processor isn't on[1].
1: http://www.osnews.com/story/27416/The_second_operating_syste...
The batteries came with a charger and were around 10 Euro each.
I always have to chuckle when that comes up.