Hacker News Reader

My 250M parameter model runs in 50ms/token ;)

Releasing April 26th when Chrome 113 hits stable. Open source NPM library you can add to any project.

That's pretty impressive. What model are you using?

FLAN-T5 Base currently, 780M parameter variant coming shortly!

What kind of model? Question-Answering? I imagine it must be quite specialised at <1b Params when many are 7b, 13b, or more?