Is there any rough rules on what model sizes will work on various RAM sizes on consumer machines?
7B: 8GB (4GB min for model only)
13B: 12GB (8GB min)
34B: 28GB (18GB min)
70B: 60GB (38GB min)
Lower bpw quants and 8-bit cache size can reduce memory requirements / expand context, but if you're sticking to mainstream L1/L2 models, just go by that.7B fits comfortably in 6GB. 70B will squeeze, into 24GB, albeit with significant (but not catastrophic) quality loss.