Both Google and Meta have done that years ago, with Tensorflow/JAX and PyTorch?

But a software abstraction only helps to reduce lock in. It does not unlock efficiencies on itself. For that new and/or hardware hardware might be needed.