Here's my naïve thinking:
· Real-time audio requires low-latency scheduling, which is unrelated to the API for audio itself, and FIFO depth monitoring so that you don't get either buffer overruns or buffers that are so full that they induce latency in excess of what your scheduling and processing latency unavoidably adds. For output, you could provide this with a /dev/audio for writing data and a /dev/audiooutputbuffersize for reading the FIFO buffer depth. For input, you could provide this by making reads from /dev/audio nonblocking, by providing a /dev/audioinputbuffersize you would read before risking a blocking read on /dev/audio, or by reading in a separate thread.
· High-performance graphics covers a lot of ground, but one of the big pain points is unnecessary memcpying on the paper path from your CPU graphics program into the GPU, whether that's textures, vertex buffers, or prerendered pixel data (though maybe we could argue that that last case is never going to be high performance.) One way that unnecessary memcpying arises is from buffers for write() that are not correctly aligned with respect to the start of a page, which unavoidably means that they will have to get copied to a place with the correct alignment. This is a big problem if you're streaming out gigabytes per second of graphics data. In particular you don't want the data that goes to the GPU concatenated with data that tells the kernel graphics driver what to do with it.
Solution: open a new file in a /dev/gfx directory for the data to be sent to the GPU, then write the data to it from a properly aligned buffer. Fork a separate thread to write this data, which is done by marking the pages COW rather than copying them. (A more radical and less sneaky solution I'm exploring for Wercam and BubbleOS is to transfer ownership of pixel data buffers from the application to the window system, causing them to disappear from the application's memory map, avoiding the need to either copy or COW them. But this definitely departs from the Unix read()/write() model.)
· A different aspect of high-performance graphics, other than raw throughput, is that you might need to synchronize your drawing with monitor refreshes to avoid an extra half-frame of latency, particularly at low refresh rates like 60 Hz. This seems straightforward to solve in a filesystem interface: reading from /dev/vbi produces a byte just before each VBI, giving you time to send any necessary new data to the GPU.
Certainly it's true that Plan 9 does not provide high-performance graphics or low-latency audio. But I don't think that proves that filesystem-based interfaces can't provide them, just that the Plan 9 designers had spent the 1980s researching DSLs, fault-tolerance, concurrency, and typesetting, not rasterization, real-time control, and animation.
In general, system-call or IPC interfaces of the form "invoke function foobar with a baz handle and a quux struct or memory buffer" can be cleanly replaced with "open /bazzes/$i/foobar and write a quux struct or memory buffer to it", can't they? I mean, that's three system calls instead of one, so about 1 μs of system-call overhead instead of 300 ns, but in the cases where that's the performance bottleneck you can usually solve it by writing N quuxes instead of one. Can't you? A potential problem arises when you need to combine multiple handles to extra-process resources in a single system call, like SCM_RIGHTS or rename(), but I don't know where those would come up in the particular application domains you identify as problematic.
But maybe my proposed solutions above won't solve the problems I think they will, or maybe I'm missing the biggest issues entirely. What am I missing?
(I'm not claiming that this approach would be easier, safer, or more discoverable than the approach of adding a bunch of system calls; Benno's talk discusses the many ingenious ways a filesystem-based API can be hard to use, and no doubt some of the same criticisms can be leveled at the above strawman proposals. I'm just saying I don't see where it inherently fails to meet performance requirements.)