But its effectively equally easy to do the same with llama.cpp, vllm or modular..
(any differences are small enough that they either shouldn't cause the human much work or can very easily be delegated to AI)
(any differences are small enough that they either shouldn't cause the human much work or can very easily be delegated to AI)
Then you want to swap models on the fly. llama-swap you say? You now get to learn a new custom yaml based config file syntax that does basically nothing that the Ollama model file already does so that you can ultimately... have the same experience as Ollama but now you've lost hours just to get back to square one.
Then you need it to start and be ready with the system reboot? Great, now you get to write some systemd services, move stuff into system-level folders, create some groups and users and poof, there goes another hour of your time.