If you're using shared libraries in your program, you already have this extra indirection - at compilation/link time the compiler/linker doesn't know where in memory the shared library will be when the program is run, so every call into a library goes through the procedure linkage table (PLT), which works by looking up the 'real' function pointer in the global offset table (GOT).

So yea, populating this at runtime depending on the CPU features supported by your CPU is possible, and the linux dynamic linker does this: http://man7.org/linux/man-pages/man8/ld.so.8.html (check out the section on hardware capabilities).