I have questions now: It seems like everything hinges on the search step, where domain-specific content is gathered to serve as prompt input for the LLM.
1. In an area like technical docs, if I can't find what I need it's often because I don't know the right terms - e.g.: I am looking for "inline subquery" but they're really called "lateral joins" - Would the search step have any likelihood of finding the right context to feed the LLM here?
2. How much value is added by feeding the search results through the LLM with the user's prompt, vs just returning the results directly to the user?
3. Are there good techniques being developed for handling citations in the LLM output? IIRC Google had this in Bard