A much lower risk strategy would be using multiple versions of llama-server to keep supporting old models that would break on newer llama.cpp versions.
A much lower risk strategy would be using multiple versions of llama-server to keep supporting old models that would break on newer llama.cpp versions.
llamacpp> ls -l \*llama\*
-rwxr-xr-x 1 root root 2505480 Aug 7 05:06 libllama.so
-rwxr-xr-x 1 root root 5092024 Aug 7 05:23 llama-server
That's a terrible excuse, Llama.cpp is just 7.5 megabytes. You can easily ship a couple copies of that. The current ollama for windows download is 700MB.I don't buy it. They're not willing to make an 700MB download a few megabytes bigger to ~730MB, but they are willing to support a fork/rewrite indefinitely (and the fork is outside of their core competency, as seen by the current issues)? What kind of decisionmaking is that?
If you include multiple version of llama, and each of those llama version depends on different GPU libraries, that could balloon the download size.
If these GPU libraries change rarely, then yes, you are correct, it might not be a problem.
It'll compile fine against latest CUDA 12.8 or 12.9 etc, but there's zero need to pack whatever the latest CUDA version is.