ZIM or not, I think the “LLM as optional sidecar” part is the right idea.
The durable asset is the knowledge base itself. A local model can be useful on top, but it should stay a layer, not become the dependency.
The durable asset is the knowledge base itself. A local model can be useful on top, but it should stay a layer, not become the dependency.
I hope we can get to the point where even a small distilled model at the 7B-30B level avoids hallucinating.