The fact that OpenAI documents theirs is already a big improvement over Anthropic. But, also, the OpenAI tokenizer got more efficient when they last updated it, rather than less. https://mdstudio.app/o200k-base-tokenizer
Interesting. New models are estimated at ~5T params, so 45,000x increase over BERT base (110m). But vocab size of 200k, so only an increase of 7x over BERT base (30k).
Where do these estimates come from?
a dark anal cavity somewhere next to marketing.
Interesting that Grok 4.5 nears or exceeds OpenAI and Anthropic models in some benchmarks at only 1.5 trillion parameters per their announcement post.
BERT is not an LLM, it's an encoder-only model, i.e. it doesn't generate new text. the first somewhat useful publicly accessible LLM was GPT-3 with 175B params, and it was also the last frontier model whose parameter count was disclosed.