I don't think you're fully considering the implications of what the outcome would look like if voter fraud was used effectively. The world wouldn't "end" because nobody would be the wiser. The whole point of cheating is to do so undetectably, so nobody realizes it and everyone agrees that the outcome is fair.
So it's our moral obligation to ensure that undetectable cheating is as difficult as humanly possible to ensure the integrity of elections. Not just say "oh, well nobody has detected anything bad so far, so it must be fine" because things would also appear fine in a severely compromised voting system.