Just run Emacs as your Lisp Virtual Machine. All it really needs is a good editor, but evil-mode is kinda serviceable.
The issue I see with the idea of lisp-all-the-way-down is that most people don’t want to write filesystems and device drivers. I know I don’t. I mean I find them fascinating, but I usually have a specific app I want to write. I don’t want to write device drivers on my way there.