But that shouldn't be a problem for tardigrades frozen in a block of ice, because they are not in direct contact with air.
Oxygen is in a high oxidation state and once reduced to water is quite stable. In order for oxygen to form from water, you'd need an oxidizer of considerable power (e.g. ozone?).