Cool use case, glad to see txtai [1] is helping (I'm the main dev for txtai).
Since you're using txtai, this article I just wrote yesterday might be helpful: https://neuml.hashnode.dev/build-rag-pipelines-with-txtai
Looks like you've received a lot of great ideas here already though!