You can add a reference to the source session for each note in the vault.
Then you could check if this is true in the actual session instead of trusting what the note says.
This is one of the ideas we used when we were building memory for Substrate.
Does your agent write to the vault automatically?