I'm running it on Serge with 51 GB of RAM. I don't think it requires any GPU memory (llama.cpp runs on the CPU), I have a RTX 2060 in the system I'm running it on. Correct me if I'm wrong.
How many tokens per second are you getting with your setup?
About .5 tokens per second if I had to guess.
Is this with the 4-bit quantization? The only issue with it is that inference is incredibly slow with it on right now, but that should be fixed up in the next few weeks I think.