My original thought here was: chill out everyone! Even if we had AGI, even ASI today... neither of those systems could know everything that you have seen, heard, dreamed, or thought. They cannot truly replace you... unless all audio, video, and eventually neuro signals in your lifetime are recorded. This is really important, as all of those things influence you to make the decisions that you make. This is what makes you unique, and important.
Then the next thought was... is that something worth fighting against? Or, should I welcome my individually trained technological replacement?
When I was talking about this with a co-worker earlier today, the thought occurred: this is not only the final frontier of privacy, but of individuality.