I think a slightly more fancy compiler could do something similar to "enforce" coop-multitasking: insert yield calls into the code where it deems necessary. While the halting problem is proven to be unsolvable for the general case, there still exists a class of programs where static analysis can prove that the program terminates (or yields). Only programs that can't be proven to yield/halt need to be treated in such fashion.
You can also just set up a watchdog timer to automatically interrupt a misbehaving program.
They forked C#/.Net, taking the async concept to its extreme and changed the exception/error model, among other things.
There are several other OS projects based on Rust, relying on the memory-safety of the language for memory-protection. Personally, I think the most interesting of those might be Theseus: <https://github.com/theseus-os/Theseus>
As long as all of them have to use an authorised llvm equivalent and that one can enforce memory access and cooperation that could look from the outside like a quite normal user experience with many programming languages available?
For performance though, I don't think halting analysis is needed. Even if the compiler can prove a piece of code terminates, it doesn't help if the inserted yield points occur too infrequently. If a piece of code is calculating the Fibonacci sequence using the naïve way, you do not want the compiler to prove it terminates, because it will terminate too slowly.