Hacker News Reader

It's possible to run with a cpu and then use system memory.

Do you know what kind of speed you can expect with inference on CPU?

Yes, around 2 tokens per second on a newer CPU with fast RAM.