This comment section is wild right now btw, no one is responding to the content, just the, i guess, misleading title.

The OS in the title is making a comparison to how page caches work. Relevant data is moved into the context the llm needs to answer chat questions. When it runs out of memory it moves a condensed, searchable version of the content to another data store. Its trying to solve the problem of relatively low max token limits.

I wonder if this makes it easier to run on small devices?