The first problem, context window size, is going to bite a lot of people.
The gotcha is that it’s a search problem. The article mentions embeddings and dot product, but that’s the most basic and naive search you can do. Search is information retrieval, and it’s a huge problem space.
You need a proper retriever that you tune for relevance. You should use a search engine for this, and have multiple features and do reranking.
That’s the only way to crack the context window problem. But the good news, is that once you do, things get much much better! You can then apply your search/retriever skills on all kinds of other problems - because search is really the backbone of all AI.