but how is the speed here? does it feel fast "enough"?

looking into to running llama on prem / private cloud but i have no idea where to start in terms of sizing, do you have any details or posts on to what the minimum / recommended hardware requirements are?

EDIT: just looked myself, not as encouraging as I'd like: "For good results, you should have at least 10GB VRAM at a minimum for the 7B model, though you can sometimes see success with 8GB VRAM. The 13B model can run on GPUs like the RTX 3090 and RTX 4090"

definitely borderline dealbreaking for solo hackers / small teams