Transporting fissile material with chemical rockets is dangerous, it's making a dirty bomb and hoping it explodes in one direction. That hasn't always worked for transporting people.
If we say there's no life on the Moon, we still plan to be there. The fallout could affect the use of the Moon, which is far more important.