A removable battery requires its own protective shell, which costs weight and thickness. When it's not user serviceable, it can just be a bag of acid resting on the main board, piggybacking on the device's own case for the required protection.
They are usually constructed of layers of battery material, wrapped in kapton tape or an equivalent, with the entire package shrink-tubed and a pigtail coming out the side for the device interface.
One could easily crush the unit by hand, the construction offers little to no protection from physical forces.