Hacker News Reader

There is a bug in llama-cpp for qwen/gemma models, use vLLM instead

what bug and it affects what?

it's a prompt cache invalidation bug that causes all input to be reprocessed instead of getting preloaded

There are other reasons to prefer vllm to llama-cpp as well