I'm working on something where I need to basically add on the order of 150,000 tokens into the knowledge base of an LLM. Finding out slowly I need to delve into training a whole ass LLM to do it. Sigh.
Can this be implemented in current opensource models?
A other option is to ask GPT to compress your tokens into a shorter prompt for itself.
[0] https://www.theverge.com/2023/4/14/23683084/openai-gpt-5-rum...