I have one of my own. My reason for this is that you can get up to 2x performance compared to llama.cpp on the same model weights. It is optimized for single model to serve it well.

Most of these custom engines exist because the more general purpose projects make contributing much harder than maintaining a custom engine (somewhat understandably). Consequently they take forever to adopt optimizations, and it is somewhat pointless if llama.cpp or vllm only reach performance parity with custom engines when the LLM architectures being optimized for are no longer the best ones.

The benefit of llama.cpp etc is just ease of use for people not already familiar with the local AI landscape.

Because with AI everybody can reinvent the wheel with ease

If you had read TFA, you would have seen that the reason is obvious.

The speed obtained by this optimized program is many times higher than the current speed reached by llama.cpp when reading weights from an SSD.

It is unlikely that llama.cpp will include any time soon code capable of reading SSDs at the speed limited by hardware.

If the SSD files are memory-mapped, you need to provide at appropriate points in the program the correct madvise hints (the Linux-specific advices, not the useless POSIX advices) to free the unused memory and populate the memory that will be soon used, in such a way as to guarantee that there will never be any page faults while reading the weights.

Otherwise, you must used asynchronous I/O with O_DIRECT, as in this project, which requires a significant reorganization of the program in comparison with using memory-mapped files, to guarantee that the desired data is always available at the right time and the inference computation overlaps perfectly with reading continuously from the SSDs.

Probably because its poorly documentrd.