Amazed to see connection oriented protocols still being used here. As someone whose been in the real time media space for decades, we switched everything to UDP about 15 years ago and achieved much higher throughput and lower latency. Our server software architectures use a single dedicated thread on a UDP port, which reads packets and distributes them to lockless queues each owned by a dedicated single thread handler exclusively bound to separate physical cores. Similarly for the outbound side. This allows us to scale vertically to maximize a single machine's resources, while allowing us to essentially cap the upper bounds on latency in high load scenarios. Our architecture enables us to have hundreds of thousands of users PER MACHINE. We've been running this architecture for over 10 years now in production successfully. While we do have logic to scale horizontally, we minimize this and use it only when its required for scaling further or traversing specific network clouds, depending on where a user is able to connect. Worth mentioning we use general linux distros on physical hardware. No virtual machines. We also use kernal bypass techniques.

Any time you add more servers to spread load, you're increasing latency because for each hop you're traversing the entire software network stack twice plus hops through hardware switches.

Nobody uses dedicate thread per client anymore (if they do, its a poor design).