This is commonly avoided by not putting memory pressure on your O_DIRECT application, which is probably an RDBMS on a dedicated single-purpose host.

Not saying more work in the space wouldn't be welcome and interesting, just that it's generally the assumption whenever I've seen O_DIRECT (since the Solaris days and through today) that what's using it is the only thing of substance on the box.