If you just keep ignoring it, eventually you will die and your estate won't be released to your heirs until the tax man gives it a final audit, and your estate will have to pay penalties and interest back to when you failed to pay.
If you're poor and/or broke, then sure, ignore an audit, because they can't get blood from a stone. :)
If you're self-employed and not reporting income...well, that's another story. But this is terrible advice that you should definitely ignore, anyways.
Like I said, my grandfather owns a business and hasn't filed in decades. It caused my Uncle some serious consternation when he took the reigns, but he never faced any consequences in the form of being held in contempt, asset seizure, etc. Just some nastygrams every now and again.
But again, this is awful advice; I was mostly being facetious. You should not just outright ignore your taxes, you should at least make a desultory effort to get SOMETHING in. But damned if I'm going to bother pulling together any more forms than are in arms/web's reach, or mail in anything more than the bare minimum.
Avoid the catastrophe.