RAFT: Sailing Llama towards better domain-specific RAG

ai.meta.com

23 pointsmfiguiere3 comments

Just because the acronym is right there on the ground doesn't mean you have to pick it up.

There's already a pretty famous Raft in computing.

I feel like this is happening quite a lot in the generative-AI space, a /lot/ of words/names that already have wide/common use in the tech-sphere are being reused for new developments in generative AI.

I'm trying to not take it as an intentional extension of the already problematic effect these same technologies are having on the effectiveness of web-search…

All the best,

-HG

Retrieval-Augmented Fine-Tuning is a really promising technique.

FTA:

> Tianjun and Shishir were looking to improve these deficiencies of RAG. They hypothesized that a student who studies the textbooks before the open-book exam would be more likely to perform better than a student who references the textbook only during the exam. Translating that back to LLMs, if a model “studied” the documents beforehand, could that improve its RAG performance?

Incidentally, the team who wrote the paper released some nice code to generate domain-specific fine-tuning datasets: https://github.com/ShishirPatil/gorilla/tree/main/raft

Reply on news.ycombinator.com