Very much not an expert here but what I understand is that most deep learning frameworks (PyTorch, Tensorflow, etc.) have some overhead associated with them just being on the graphics card. This takes PyTorch code and removes the overhead by translating the network into a "native" language for the card (CUDA for NVIDIA).

What I'm not sure is what "HIP" is in this context.

The way I'm reading this is it's the difference between running code in an interpreter vs. on the bare metal (for the GPU)