How do you manage scope creep (ie, context size), and contradictory information in the context?
For contradictory or stale information, since these are based on emails and conversations, we use the timestamp of the conversation to determine the latest information when updating the corresponding note. The agent operates on that current state.
That said, handling contradictions more explicitly is something we’re thinking about. For example, flagging conflicting updates for the user to manually review and resolve. Appreciate you raising it.
That's a great idea. The inconsistencies in a given graph are just where attention is needed. Like an internal semantic diff. If you aim it at values it becomes a hypocrisy or moral complexity detector.
His manual approach, which won't work if applied directly ( or more specifically, it will, but it would be unnecessarily labor intensive and on big enough set prohibitively so ), because it would require constant filtering re-evaluating all emails, can still be done though.
As for exact approach, its a slightly longer answer, because it is a mix of small things.
Since I try to track, which llm excel at which task ( and assign tasks based on those tracking scores ). It may seem irrelevant at first, but small things like: 'can it handle structured json' rubric will make a difference.
Then we get to the personas that process the request, and those may make a difference in a corporate environment. Again, as silly as its sounds, you want to effectively have a Dwight and Jim ( yes, it is an office reference ) looking at those ( more if you have a use case that requires more complex lens crafting ) as will both be looking for different things. Jim and Dwight would add their comments noting the sender, what they seem to try to do and issues they noted ( if any ).
Notes from Jim and Dwight for a given message is passed to a third persona, which will attempt to reconcile it noting discrepancies between Jim and Dwight and checking against other like notes.
...and so it goes.
As for flagging itself, that is a huge topic just by itself. That said, at least in its current iteration, I am not trying to do anything fancy. Right now, it is almost literally, if you see something contradictory ( X said Y then, X says Y now ), show it in a summary. It doesn't solve for multiple email accounts, personas or anything like that.
Anyway, hope it helps.