Could you elaborate? It may not be a replacement, but I can think of a few use cases where RAG could be avoided entirely by having the model be fine tuned on the right dataset.
A model using RAG can tell you why it answered a certain way, and cite chapter and verse from the underlying data. That's impossible just from model weights.
RAG is straightforward but i improve it by generating extra keywords to help the search. I also add extra metadata filtering to return both technical documentation and more fluffy sales and marketing stuff. IMO, semantic search alone might not be the best, and I want to also add keyword search.
I think the thing most people will struggle with is prepping the documents. Chunking them to the best token count for your use case for example.
I'm currently trying to build a hobby virtual assistant, and it looks like RAG does need some nuance to implement correctly (from what you said). A library that does all of what you mentioned would take a lot of work out of it.
a) run locally through a web ui b) run with a local vectorstore
From my perspective of someone using it as a bot that runs in a container using a cloud based vector store makes a lot of sense, but yeah that's really good feedback and not something I considered. Hrmmm. Ok let me get a look at some local vectorstores.
It works as a library, you can require the PIP package and get started immediately.