Totally agree. Really rich people don't do their own taxes. They have armies of professionals doing that for them. The professionals don't get the "overlooking" defense, and neither should the person hiring them.
Require the tax lawyers to formally sign off on the validity of the returns, and give them some form of liability or consequence if the dubious tax shelters they recommend turn out to be plain old tax cheatin'.