Hacker News Reader

Does this run on CPU or GPU?

Would it be advantageous to divide the load and run on both?

Obviously cpu

Not obvious to me. I thought these AI things required vast amount of GPU power. Or is that for the training part?

Inference works fine on CPUs while a bit slow but fine for a chatbot

No. Ollama takes advantage of Metal on Mac M1.