and you can throttle with ionice on xen running a Linux dom0[1]. It works pretty great if you have some user who is going overboard and you want to make it suck for just them and suck less for everyone else.

It doesn't work like the CPU scheduler; which is to say, I can't set ionice on all the guests and expect good performance on the light-use guests while a heavy-use guest is thrashing the disk. (I can expect the good performance for the light-use guests when the resource in contention is CPU time.) It's possible that either ionice isn't a good enough tool, or that I don't know enough about it... but I think it's far more likely that the variable nature of disk throughput makes balancing requests harder... (e.g. how many IOPS a second I get varies a lot from second to second, based on where the sectors I want to operate on are in relation to the read/write head. How much disk bandwidth I get varies even more.)

I am pretty curious about how much this situation will change with SSD storage.

[1]http://wiki.prgmr.com/mediawiki/index.php/Chapter_7:_Hosting...