HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by senseiV | Hacker News Reader
Parent
Full thread
senseiV
·
GPT 2 and 3 used the p50K right? Then GPT-4 used cl100K
View on HN
lifthrasiir
·
Yeah, see [1].
[1]
https://github.com/openai/tiktoken/blob/main/tiktoken/model....
Reply on news.ycombinator.com