This is an outstanding idea, I love it!

It would be great if you could support Ollama (or an OpenAI compatible endpoint) for private LLMs.