All of this stuff seems like a band-aid solution. These things need to be trained ground-up to maintain and update persistent memory (maybe outside the context window?).

Also seems like a requirement for any sort of continual learning capabilities as well.