For those who aren't able/willing to use a fully static version of the site, Varnish (or Apache Traffic Server, Squid, or Apache or Nginx's Cache mode) can provide a huge improvement in performance, and (definitely with Varnish and a custom VCL, possibly not as easy with others) can even handle stuff like grace mode (i.e. return cached, expired results if the origin is offline) and purging cached objects when they're updated in the WP DB (via a plugin which communicates with Varnish).
Im not sure what you mean about maintenance problems?