RAG architecture for SaaS – Learnings from building an AI code assistant

For anyone confused what "RAG" is: it stands for "Retrieval Augmented Generation", which is where the LLM model is paired with some sort of "database" (often a vector database) where LLM looks up additional data when performing the task.