I’m neither a Node dev nor an AI person, though, so that probably helps.
I stand corrected. 8 GiB might be enough IFF you're not doing heavy multi-tasking during development, and you're not coding in JS or its offshoots.
I will say the comments about "the SSD is so fast you won't notice" is true from an end-user perspective, in that I experienced no visible lag on any app. This may not be true with the smaller-capacity M2 models, since they have slower SSDs than the base M1s.
It very much depends on the size of the document and how long it’s been open. I’ve actually had Gdocs be over 3gb for a single document in Safari before.
A 7B model @ Q4 needs roughly 6.5GB of memory.
I have a 64GB M1. If my memory pressure is high and I start up a model that goes into swap, the machine becomes unbearably slow.
Mistral 7B q4 seems to use ~ 5Gb VRAM
A few months ago I could not run Llama 7B q4 on my laptop's rtx 3070 (8gb VRAM), but Mistral 7B it runs very well and still has approx 2gb VRAM left even with a big context.
Not sure if this is an optimization on inference algorithms or if Mistral 7B is just more efficient
...I am not an OSX person, but if you can boot headless, it can squeeze 7B on there with full context