Even "compiled" JAX or PyTorch can leave some performance even if you hit the common path (It's also "praying" that the compiler actually works if you do anything non-standard).

But memory wise, there is almost no optimization or reuse (and it's sacrificed for performance), which leads to insane memory usage.

And it's not that they are bad - but optimal compilation of a graph is combinatorially explosive problem, impossible without heuristics and guesswork (what to reuse and waste memory vs recompute). A good programmer can do a significantly better job.