Cool idea. Hard problem. To do this properly you need access to all communications of all family members. iMessage and email would be needed for my family to really take advantage of this. That level of access would demand more trust than I'm comfortable giving to a third party like this.
Skylight is making some progress as a family hub of operations. I could see value in something like this in that ecosystem, especially if there was a way to control data sharing.
For example, could this run on my Mac, review all the data, then create a memory/fact feed that I can review and control?