Hacker News Reader

I wish projects like these would mention about the minimum GPU VRAM one needs, for inference or fine-tuning.

This is simply a finetune of LLaMA-13B, so it has the same ~11GB RAM or VRAM requirement as 13B for inferencing.

With 4bit LoRAs you can finetune 13B with 12GB of VRAM. On a 3090 it can take as little as a couple of hours.