Hacker News Reader

Comment by throwawaysea | Hacker News Reader

Are there parallels to the cargo pile up in computing? For example, I have seen some past stories here where a software system has an outage and then has trouble returning to a healthy state due to all the congestion from other systems attempting to retry everything that didn't succeed during the outage. It seems like absorbing such a disruption in the world of physical infrastructure would be much harder.

View on HN

Yes. If you want a theoretical treatment of it, read up on queueing theory, which basically arises anytime demand outpaces supply.

This arises in many areas in CS with many different names. Queues, buffers, backpressure, throttling, retries, DDoSing, congestion control, scheduling, QoS, pipelining, async-await/callbacks/channels/continuations, etc. are all mechanisms for dealing with various properties of this phenomenon. OSs and networks in particular are rich with this, as their normal state is for demand to outnumber supply (more connections than interfaces, more FDs than disks, more processes/threads than CPUs, …). As you mentioned, distributed systems also care about this at a higher level.

This is typically referred to as backpressure. For example a failure can break your messaging infrastructure and cause producer buffers to fill up and fail causing data loss. Once the infra comes back online it can essentially be DDoSed by all the services trying to clear their buffers/reconnect/resync

You can mitigate it with stuff like exponential back off (from the client/producer side) or rate limiting. It’s a complicated problem though.

You will get this with malnourished mail servers. If your queues arent flowing, mail will pile up. If the disk becomes full, you will now not only have to clear up the original issue but also scavenge some disk space so the transport can work. Though that might just be an exchange-ism.

Spinning up extra capacity to handle a backlog or growing queue also has physical parallels, it's just harder / costs infinitely more when not in use.

Gathering more data than you have capacity to process/store.

Reply on news.ycombinator.com