I know the whole human flight is different to bird flight argument, so I guess we'll just have to wait and see how it all plays out. Exciting times :)
If the technology stagnates, the hardware stops improving, and the software stops getting better, then we would hit the limit pretty quickly.
> After all humans don't drive by analysing 1 frame at a time, we have a model of how the world works, we anticipate, we have intuition etc...
All self driving cars also maintain those models as well, and use frame by frame input to correct those models in real time (but they have already guessed where something is going to be 100+ frames from now).