Hacker News Reader

They have published the inference code [1], you can’t just throw their model at llama and expect it to work.

[1] https://github.com/togethercomputer/HybridAgent

Yes, that's apparent now. But, many Qwen-derived models run fine without any changes to llama.cpp, and there are gguf quantizations on HuggingFace, so I thought I'd give it a try.

Reply on news.ycombinator.com