> We can understand this with the help of Little’s law from queuing theory. It states that the average number of requests in the system (Qd for queue depth) is equal to the average arrival rate of requests (throughput T) multiplied by the average amount of time to serve a request (latency L).

First off, this is obviously false. If you can serve 9req/s and you're getting 10req/s the size of the queue depth is growing at a rate of 1req/s. It's not stationary.

Second, what's the connection between this and gpus? What's the queue? What's the queue depth? What are the requests?

Seems to me that the article focuses more on being smart than actually learning.