I'm not familiar with the AIO interface, but it sounds fairly handy. Could someone comment on what he means by it not having 'many nice aspects'?
I'm not familiar with the AIO interface, but it sounds fairly handy. Could someone comment on what he means by it not having 'many nice aspects'?
- You should always use O_DIRECT or it defeats the purpose. Consequently, the caching and buffering you normally get for "free" using more conventional approaches is not available. For good performance you have to write your own disk buffer cache implementation which is not trivial if you have never done it before. This buffer code will exceed the size of the AIO code if you are doing it well. On the upside, for many purposes you can design a buffer cache that greatly outperforms the standard kernel implementation.
- You need to minimize the number of file system metadata operations that can block and/or bypass the file system altogether. I recommend the former. In practical systems, you end up doing very coarse file system operations, like allocating a huge chunks of disk, and then building your own lightweight disk management system on top of that. Again, this is a lot of extra work for the sake of using AIO effectively.
- You need to figure out the best way to schedule all of the disk I/O operations you are now responsible for. Background writing, prefetching, etc that are used to optimize disk performance are now part of your implementation.
Linux AIO is simple and pretty nice in the abstract. Using it well requires some complex domain expertise around scheduling, file organization, and cache replacement algorithms and techniques. There are no libraries that provide quality implementations of the supporting infrastructure that I am aware of so you have to roll your own. You can't do just a little bit of AIO and see much benefit in most cases.
There are big benefits though which is why advanced database engines are implemented this way. If you build out an entire userspace disk I/O subsystem on top Linux AIO you can easily double your disk I/O performance or more relative to using the conventional POSIX, mmap(), etc interfaces. Furthermore, it often can be designed to degrade performance more gracefully during periods of high I/O pressure.
Unless a program is designed to use it, it won't be. It doesn't work on sockets or pipes and filesystem support is limited.
The only things I have seen that can use it are MySQL, DB2, and Oracle.