> The major breakpoint would be when libraries like Numpy do it natively
That already happened [0]. NVIDIA has a 1:1 replacement for Numpy called CuPy that does this and is what powers their RAPIDS framework (which is a 1:1 replacement for Pandas that runs on GPUs).
Some people were complaining in [0] about CuPy reproducing numpy's bugs..