I agree with you. One “what if” possibility that I think about from time to time is an open source community back in the 1990s rallying behind an OS built from Smalltalk or a language environment offering similar capabilities such as Self or Common Lisp. I figure a Pentium or PowerPC personal computer from circa 1994 should have enough power to drive an entire ecosystem of applications around Smalltalk, especially with the compiler and virtual machine innovations the Stanford/Sun researchers working on Self made. The environment’s introspection and metaprogramming capabilities would have resulted in a highly flexible, extensible system, but with a simpler and cleaner design than contemporary Windows and Unix desktops, which have many layers of complexity.
Recently I’ve thought about making a Smalltalk- or Common Lisp-based operating system with the features you mentioned, but I ended up changing direction when dealing with the problem of interoperability among different languages. As appealing as Smalltalk and Common Lisp are, many people prefer different languages, and unfortunately different languages have different semantics, making interoperability difficult. I prefer a protocol-based approach to interoperability. I’m right now experimenting with Plan 9, an operating system designed by many of the same people who worked on Unix at Bell Labs, and seeing how I can implement reflection and metaprogramming through Plan 9’s “everything is a file” facilities, and also how I can take advantage of the 9P protocol to enable component-based software design. Some more of my ideas can be found at www.mallowos.com.