HNHacker News
TopNewBestAskShowJobs

bauefi

2 karma · joined February 26, 2023

submissionscomments
bauefi··on Evaluating modular RAG with reasoning models
It depends on how you do retrieval. If you just use dense embeddings for example you can get the latency of one search query down to maybe something like 400ms. In that case multiple sequential look ups would be ok but your embeddings need to be good enough of course