That's neat. What I'd worry about is boot-up time. I let people compile and test their program with an AJAX button in the middle of taking a test. It's already 3-4 seconds before they get the result back, more delay would hurt.

With tasks queuing up as they already are, I might not be able to keep a fresh VM pre-warmed every time, either.

edit: The other factor is of course the prospective cost of a brand new configuration. Unless I find I can't prevent the current config from being breached and ephemeral VMs will fix it.