Hacker News Reader

Devs confirmed that the small ones use 800B, 1.5T is for the large ones

@thunderbird120 asked a Stability employee and say that the plan is going to keep training the models up to 1.5T. So I don't know where do you read this.

https://github.com/Stability-AI/StableLM#stablelm-alpha shows that the 3b and 7B had 800b training tokens.

That may be, but the weights you can download today were trained on 800B

yes of course that's why they use "will be trained" on the GH repo.

I think they are “checkpoint” models in this case.

Will be fun to compare when completed!

Are not all models checkpoints? I think you may be interpreting it too colloquially.

Reply on news.ycombinator.com