“If your previous employer sues you they will lose” is not the counter you think it is. You were lucky they didn’t litigate and luckier still that they didn’t consider a non-compete of outsized importance like that Wisconsin company does: Some employers are extremely litigious, and will try to exhaust you in court because they believe they’re protecting internal secrets.
That sort of thing must not be discounted when considering what advice is appropriate in this kind of situation—and especially what the overall policy of a society should be: If the company I referred to above were a California company and you were an employee in California, “they will lose” and “you won’t spend millions” (or at least “you’d recover costs”) would actually be accurate.