You could develop one universal way to create and deploy functions to different languages, but now you've introduced one more language to get people to use.
And in practical terms, migrating this sort of thing from one language to another when necessary is generally a good enough solution.
Imagine a high-level, straightforward imperative language X whose only purpose is to specify libraries and data structures. This X has strong typing, memory safety and maybe also memory alignment directives but abstracts away as much as possible from any actual environment/platform/CPU layout. It has a precise semantics and even some ways of specifying pragmatics (e.g. big O runtime behavior of functions). In X you write "pure" algorithms that do not access (m)any OS-dependent features. Libraries in this language are then transpiled to various target languages and platforms.
To use all libraries that have ever been written for X, someone only needs to write a transpiler-backend for the target language/platform once.
LLVM intermediate language is similar but too low-level. Think about a high-level counterpart.
This approach is used to some degree in producing, for example, optimized BLAS implementations and other similar routines:
http://view.eecs.berkeley.edu/wiki/Autotuners
Of course, you do then need to juggle multiple languages. Different languages can be better at expressing different things, though (e.g., "what the function does" versus "how the function does it"), so it's not inherently crazy.