The X11 Files

I never understood the confusion over how X11 refers to the "server" and the "client". You hit a key in xterm, the client, it requests, to the server, that certain pixels light up in such a way as to represent a character. Clients make requests, servers carry them out. The apparent confusion is more confounding when you consider that, today, few people actually run X clients remotely for everyday purposes, so the server and the client are right in front of you on the same machine, so confusion over the "server" being a "remote" resource are unfounded. On top of that, I can not comprehend what kinds of requests could be made by the video hardware, and the software that drives it, that would make it a client and the X application the server.