Hacker News Reader

I did not have great luck with 405b on an M1 Max Mac Studio with 64 GB of RAM via olama. I think memory was this issue. 70b runs with acceptable speed

View on HN

64GB of ram is 550b bits of ram. That doesn't give you a lot of options on how many bits you want to keep per model parameter.

Reply on news.ycombinator.com