And at the risk of being a downer, I'm not sure Open Source LLMs is actually viable.

Training a model requires two or three orders of magnitude more investment than the typical OSS/Creative Common contributor can reasonably afford.

It reminds me a bit of Open Source hardware (CPU, GPU).