Hacker News Reader

This scenario already works that way. The very first sentence says "servers, each of which can only handle a single concurrent request, and has no internal queuing". This implies that the load balancer waits for a server to finish a request then immediately sends the next one.

View on HN

I don't believe it does. As I understand it, the load balancer has a queue in which it can buffer infinite requests, but it drains that queue by pushing work to the backend servers in what's probably a round-robin fashion. So there is secondary queueing at each server. Even the "least connections" strategies available through some load balancers do not usually behave as you might expect (by always sending the next request to a server that's idle). Pull-based load balancing via a queue has its own downsides but the big upside is to make latency essentially a constant low overhead regardless of the number of servers in the typical case.

I think your imagination decided to rapidly overcomplicate what is literally (literally literally) Queuing Theory 101 example

If I were to guess there weren't any "backend servers" at all. It was just array of random increasing numbers (that stand for request arrival times) and arrays of numbers with minimum distance (that stand for time each consumer took a request)

there's no connections to "least-ify" the strategy about. There's no difference between consumers, no matter the amount of requests having been processed

Reply on news.ycombinator.com