One of the biggest issues people bring up when someone asks "Why can't the OS handle garbage?" is that the OS doesn't know what is garbage and what isn't. But if you only ran languages that ran on the OSes virtual machine, then it would know the memory model. Java and Racket both have histories of languages being built on top of them, so if you don't like the base language you could create your own.
I'm not saying this would be practical or useful (other than maybe as a proof of concept) but it could be fun!