LLMs can 'search' thru previous prompts. If they don't fit on GPT3's token limit, I've seen apps use OpenAI embeddings model for this. It works to the point that it feels like it 'remembers.' Granted that even embeddings will have a cap.
Do you have a source on this? Would love to read more about it, I was under the impression it just "absorbed" the previous prompts into the current until token limit has been reached.