HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by wmf | Hacker News Reader
Parent
Full thread
wmf
·
I would expect a 40GB GPU to run 35B models.
View on HN
karolist
·
35B models need about 22G vram with Q6 quantization, you should be able to run 70B models with low Q
Reply on news.ycombinator.com