Hacker News Reader

I have solved this problem by forcing the model to compress every time by specifying token budgets for the documents

You mean word budget? I don't see how a token budget on the LLM helps with doc length. Or do we denominate the length of text docs in tokens now?