first, this is amazing!
second, how does one increase the context window without requiring obscene amounts of RAM? we're really hitting the limitations of the transformer architecture's quadratic scaling...
second, how does one increase the context window without requiring obscene amounts of RAM? we're really hitting the limitations of the transformer architecture's quadratic scaling...
[1] https://research.google/blog/chain-of-agents-large-language-...