Maybe I misunderstand, but it seems like they are using LoRa, which is a fine tuning implementation. That requires an already existing trained LLM. If that's true, I think that the title of this submission is inaccurate, as this doesn't let you train a model from scratch with 2 consumer GPUs.