This is because the Haber process, necessary for fertilizer, involves industrial quantities of forced air, and the USSR put adapters on their Haber process plants for gathering inert gases for titanium welding back in the 1970's, and those are now used to gather materials for semiconductors around the world. I don't know enough whether US or other Haber plants could also be adapted to gather up these noble gases, but presumably it would be more expensive than using what the Ukraine provided, or else it would have been done before.
That is one issue among many, obviously, but it suggests the scale of the problem for making all domestic supply chains.