LongChat: Train and evaluate long-context LLM-based chat bots

github.com

3 pointsjchook3 comments

Are there models available that have been fine tuned in this way? I don't have much interest training on my own data but (like many I'm sure) would love a Vicuna-like model that had an expanded context. Has anyone found or hand any success with such a thing?

It seems that the Vicuna folks have trained some LongChat weights, e.g. https://huggingface.co/lmsys/longchat-7b-16k

Additionally, there here some long context 30B models on huggigface now. For instance:

https://huggingface.co/TheBloke/GPlatty-30B-SuperHOT-8K-GGML

https://huggingface.co/TheBloke/Vicuna-33B-1-3-SuperHOT-8K-G...

These will work with 8K context on even a modest system, thanks to Koboldcpp. 16K context is more exotic atm.

Reply on news.ycombinator.com