HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by armchairhacker | Hacker News Reader
Full thread
armchairhacker
·
Current speed is “approximately one third of a token per second”
View on HN
marcobambini
·
Right, we trade speed for the ability to run a 2.7T-parameter model while preserving accuracy. It is a first version, and we plan to improve the inference performance.
Reply on news.ycombinator.com