pacjam

Interesting analogy! There's no clear infinite tape equivalent in MemGPT, but you can view the virtual context loosely as a tape. Moving the head could correspond to MemGPT indexing into virtual context - if the data is in-context (inside the LLM window), it's a direct read, but if data isn't there (it may/may not be stored in external context), the read requires a function call to "page in" the data.

Awesome, feel free to open issues or PRs to our repo if you want to contribute! It's all open source and under Apache 2.0, and we're actively looking at integrating common workflows to the CLI.

You're correct that MemGPT doesn't do editing of LLM weights like in ROME - the "memory" we're considering in MemGPT is at the text/token level, not the weight level. The core concepts behind MemGPT is giving the LLM the ability to edit a working memory scratchpad (held in-context) and reading/writing to external context via functions. An important detail is that reads are always paginated (chunked) to deal with finite context limits, and MemGPT can do many iterative read/writes from a single user input (by chaining functions together). This allows MemGPT to search over a large database of documents for example, collecting information from various sources to return an answer (as in our LlamaIndex API docs example on the README).

If you have a chance try it out via the Discord bot or with the GitHub repo! Or even just check out the short demo GIFs we released (at https://github.com/cpacker/memgpt) to get an idea of the MemGPT inputs/outputs.

The high-level memory read/writes are quite intuitive and you may be surprised at how closely it matches what you were suggesting to your friend.

Thanks for your interest! Question - does the title of the chat ever change after it's first assigned? If so, using a recursive summary to refresh the title sounds like a reasonable idea (especially if you're already computing a summary to extend context).

From what I remember the title in ChatGPT gets set once after a few messages, in which case I'd assume it's generated with a special "title generation" prompt (that gets the first few messages as input).

In either case since I don't work at OpenAI I can't tell you for sure ;)

Happy to chat more about other ideas in this direction! There are plenty of things we tried with varying degrees of success (especially when trying to get MemGPT to work on less powerful LLMs), and we'd be interested in hearing what you observed in your own work.

Recursive summarization is a simple and popular way to provide the illusion of infinite context (when you need to free up space, just summarize the oldest N messages into 1 summary message). It's lossy and you'll inevitably lose important information, but it should degrade relatively gracefully. In MemGPT we use (implicit) recursive summarization on top of all the explicit memory management.

Hey all, MemGPT authors here! Happy to answer any questions about the implementation.

If you want to try it out yourself, we have a Discord bot up-and-running on the MemGPT Discord server (https://discord.gg/9GEQrxmVyE) where you can see the memory editing in action - as you chat with the bot, you'll see MemGPT edit its memory to update its profile about you (and itself).

Everything's open source, so can also try running MemGPT locally using the code here: https://github.com/cpacker/MemGPT. In the repo we also have a document-focused example where you can chat with MemGPT about the LlamaIndex API docs.

Thanks @upghost

This is similar to how our external context is implemented under the hood - you might be interested in our perpetual chat bot example in the GitHub repo (https://github.com/cpacker/MemGPT), the message traces in the demo GIF should give you an idea of how things work at a glance.

If you want to try the MemGPT Discord chatbot, join the Discord server (linked above), then check out the #memgpt channel to start messaging the bot.

If you want to run the chatbot or API docs examples locally, you can follow the instructions here: https://github.com/cpacker/MemGPT/tree/main#quick-setup.

Update - we just released a Discord perpetual chatbot implemented on top of MemGPT, you can try it here: https://discord.gg/9GEQrxmVyE

You can also run the chatbot demo + a doc QA bot demo (where you can ask MemGPT about API docs) locally with the code on GitHub.

Thanks for your feedback @skrebbel. We called it MemGPT - in reference to teaching LLMs how to manage their own memory hierarchy. The idea and title evolved when we realized we were teaching the LLM to read/write, and this enabled perpetual chatbots, doc QA, etc. Various capabilities by a few simple abstractions, inspired by the original unix paper's 4 key abstractions - read/write/open/close. Further, in the title we say "Towards", indicating a qualified step in that direction.

Thanks for your interest! Left a comment on the other thread

Explicit memory management (MemGPT-style) vs implicit/external memory management is an interesting tradeoff. Like you said, adding all the instructions on how to manage memory consumes ~1k tokens (using the default prompts on our MemGPT GitHub release), which is a lot when your context window is 8k. Additionally, it requires the base LLM to be very good at instruction following; gpt-4 can do it well, but it's much more difficult to get explicit memory management to work with gpt-3.5-turbo or llama2 70b finetunes (so to build a robust system, you may have to end up having to "split" the thinking out of necessity).

One of the main benefits of explicit memory management is simplicity - e.g., you don't have to manage logic between a "memory creation" thread and a "dialogue thread". The explicit approach also integrates well with the iterative paging/retrieval for document analysis we demo in the paper/on GitHub.

Hey OP, thank you for the post. In this work, we are looking at one particular functionality of an OS - memory management. The project started off motivated by trying to extend context length for a project we were working on. The analogy to OSes came naturally as the project progressed and there was growing similarity to the caching and memory hierarchy relation.

Thanks for checking out our work! Yeah, great point, this is even more critical in those scenarios when you have very limited memory. You can play around with different context sizes using the code on GitHub: https://github.com/cpacker/MemGPT/blob/main/memgpt/constants...

Hey all, I’m the lead author on this work. Thanks to the OP for posting, and for all the interest and feedback. To clarify a few points: the goal of this work is to investigate the extent to which an LLM can manage memory and different memory hierarchies, applying lessons from operating systems to extend effective context lengths. All of our code is open sourced at https://github.com/cpacker/MemGPT so you can try it out yourselves!