So going after people who need to build low latency high-throughput inference systems.
Also as someone else pointed out, they also target all kinds of hardware, not just NVidia.
So going after people who need to build low latency high-throughput inference systems.
Also as someone else pointed out, they also target all kinds of hardware, not just NVidia.
The primary advantage of mojo seems to be Gil-free syntax that is as close to Python as possible.
In Mojo it's pretty much the whole point of the language. If you're only using CPUs, then yeah, PyO3 is a good choice.
In the same way that LLVM allows CPU code to target more than one CPU architecture, MLIR/Mojo allows GPU code to target multiple vendor's GPUs.
There is some effort required to write the backend for a new GPU architecture, and Lattner has discussed it taking about two months for them to bring up H100 support.
They also miss CPUs on Windows, unless using WSL.