Hacker News Reader

> For people without a 24GB RAM video card, I've got an 8GB RAM one running

What're you using for this? llama.cpp? Have a 12GB card (rtx 4070) i'd like to try it on.

I believe its just a HTTP wrapper and terminal wrapper around llama.cpp with some modifications/fork.

Does ollama have support for cpu offloading?

> Does ollama have support for cpu offloading?

Yes.

A perfect blend of LMGTFY and helpfulness. :)

lol. I try not to be a total asshole, it sometime even works! :)

Good luck to you mate with your life :)