Isn't this usually done by having something that takes in a query, finds possibly relevant info in a database, and adds it to the LLM prompt? That allows the use of very large databases without trying to store them inside the LLM weights.