Hacker News Reader

> Rocket 3b uses a slightly different prompt syntax.

Wouldn't it be better if llamafile were to standardize the prompt syntax across models?

There's currently no standard because there's no one objective best way of handling prompt syntax.

There's some libraries which use the OpenAI API syntax as a higher-level abstraction, but for the lower-level precompiled binaries used in in this post that's too much.

Yes there is...HF chat templates is something that is being standardized on, slowly.

It's just a jinja template embedded in the tokenizer that the model creator can include.

llamafile can provide an abstraction, but ultimately it boils down to how the model was trained and/or fine-tuned.

Reply on news.ycombinator.com