Hacker News Reader

I'm running a Ryzen 7 8745HS with Radeon 780M (27GB RAM), using Ollama's Vulkan backend.

I get ~45 tokens/s generation and ~680 tokens/s prompt processing.

The problem is ollama is not caching the prompt, I get -> forcing full prompt re-processing due to lack of cache data

So this wouldn't work in its current state in an agentic loop.

What a pain to be forced to use dark mode...

Thanks. I'll seek an alternative

Ollama is generally just not great software. Use something else and you likely won't have that issue.