Destroying one career is not the same as destroying a field. Mathematicians like to believe the likely apocryphal tale that Nobel didn't endow a prize for math because the likely first winner was sleeping with his wife. Nevertheless, the Fields medal distorts math far less than Nobel prizes distort the sciences.
Mathematicians pursue math in defiance of the far greater rewards of other paths. I know at least nine Fields medalists and many near misses. Unlike the Nobel prize, neither the quest nor the realization of the Fields medal turned anyone into an ashpile.