People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable (gpt3.5+4/palm/claude)…
There are only a few hundred folks who have actual experience training at this scale.
llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.