"Maybe it's just familiarity but even that feels too simple an explanation"
I think it is. Because it has always been like this for anyone since he played the first time. You press the key and (a bit later) music comes. I doubt anyone consciously notice that it is not instant, but your mind notices and adjusts to it.