blackcat786

The need for HA software to remap ip's should not be there if the service has been designed in a stateless manner. As soon as a service goes down the balancer should send the next request to the next node.

In addition - At the entrance of the cloud(external ip's of the balancer) - I would suggest dnsmadeasy that can do round robin dns along with dns monitoring with a low ttl of 5 minutes that in case of failure can change your A records.

So the service should be designed with not having to rely on ip re-assignments.

Exactly. There is always a "deployed system limit" even in horizontally scaling DB layers as deployed with sharding etc.

The challenge is that by the time you reach the 300th request your system is unusable. In the example mentioned the system starts to experience exponential degradation around the 100th request. One wants to avoid reaching that state of the system where the exponential degradation starts . There IS a better solution to this than just rejecting the request and it has to do with request queuing.