Maybe he has this in mind:

Imagine a high-level, straightforward imperative language X whose only purpose is to specify libraries and data structures. This X has strong typing, memory safety and maybe also memory alignment directives but abstracts away as much as possible from any actual environment/platform/CPU layout. It has a precise semantics and even some ways of specifying pragmatics (e.g. big O runtime behavior of functions). In X you write "pure" algorithms that do not access (m)any OS-dependent features. Libraries in this language are then transpiled to various target languages and platforms.

To use all libraries that have ever been written for X, someone only needs to write a transpiler-backend for the target language/platform once.

LLVM intermediate language is similar but too low-level. Think about a high-level counterpart.

Just guessing at the parent's intent, but the specification of a function can be decoupled from the production of an optimized implementation of that function, especially if the structure of the function is constrained in some way.

This approach is used to some degree in producing, for example, optimized BLAS implementations and other similar routines:

http://view.eecs.berkeley.edu/wiki/Autotuners

Of course, you do then need to juggle multiple languages. Different languages can be better at expressing different things, though (e.g., "what the function does" versus "how the function does it"), so it's not inherently crazy.