I think this can all be boiled down to "memory", and I think the only thing restraining it is that these models don't retrain and store full reweighted snapshots after each move... so basically a storage, not a processing bottleneck. I'd predict the next major innovation in the space will be some type of NN quick diff that lets you restore a memory and personality right into the weights, from a relatively small file without having to go through a whole training process and dialog every time you boot it. Right now it's like getting back to level 8 of Contra... or level 100 of Flappy Bird... but it's going to be a few simple hacks to jump it to the point you want it. Then there's memory, de-falsification, and what we'd describe as "personality".