Wow thanks for sharing that’s such a clever hack

For 1. I think you may find work with semantic text chunking interesting, and the possibility of overlaying embeddings with one another (e.g. powering fragen.co.uk youtube search, the transcriptions whenever you click the "play" icon are semantically chunked...). Audacity has a good Autochapter API for this too. I found after good OCR postprocessing, semantic splitting is most important.

For 2 - have you used this or found it to work? I found asymmetric embeddings like MS MARCO trained models much superior when you have a reasonably bespoke corpus (like the demos of yours I've seen) - because, TLDR, it had a negative impact on recall, even as it improved Precision - if your GPT hypothetical answer is a misconception or joke answer, you'll be semantic searching joke answers, etc - same with numbers, GPT-3 might suggest short or plurality-default answer, like US phone numbers for "what is head office of marketings number?" when you have UK numbers in your corpus you would prefer to be surfaced.

I’ve done some experiments and the ideal seems to be semantic chunks of max 175 tokens. But only if those chunks are already very dense (use summarisation to get there)