I'm not sure I entirely follow what he's saying. My understanding is that Von Neumann is saying that at its core, mathematics is about describing nature, so the mathematician should prioritize keeping those descriptions useful over rigorous. This isn't an attempt to summarize the whole article, just to see if I'm in the ballpark on what he's saying.