according to Facebook Llama beats GPT3 on multiple benchmarks with smaller models that can be fine tune on a single A100 GPU

EDIT: correcting the type of GPU