Hacker News Reader

Comment by brucethemoose2 | Hacker News Reader

We've been testing it in the local llm Discords, turns out its just a llama 7B finetune that can run on any old GPU (which is cool).

https://huggingface.co/brucethemoose/LargeWorldModel_LWM-Tex...

https://huggingface.co/dranger003/LWM-Text-Chat-128K-iMat.GG...

And its long context recall is quite good! We've already kind of discovered this with Yi, but there are some things one can do with a mega context that you just can't get with RAG.

View on HN

> but there are some things one can do with a mega context that you just can't get with RAG.

Can you elaborate? In my mind, RAG and "mega context" are orthogonal - RAG is something done by adding documents to the context for the LLM to reference, and "mega context" is just having a big context. No?

I think he means not needing to have a great search system to identify rag chunks. Just throw everything in.

Or more specifically, be able to look at everything all at once instead of in chunks.

> And its long context recall is quite good! We've already kind of discovered this with Yi, but there are some things one can do with a mega context that you just can't get with RAG.

I've got to imagine that a mega-context like this can help RAG work in ways that just isn't possible otherwise. i.e. bring in many more search results or surrounding context around the results so that the processing can do much more.

Reply on news.ycombinator.com