- shit is not working
- is this an attack?
- no it's us
- how?
- that's how
- let's go back
- have to get supervisor
- roll back huge thing
really that long?
- shit is not working
- is this an attack?
- no it's us
- how?
- that's how
- let's go back
- have to get supervisor
- roll back huge thing
really that long?
- monitoring system picks up irregularity (smoothed over some window of time, which delays alerting)
- alert propagates to humans
- humans may take time to notice alert (even a page takes a few seconds to read)
- humans make decisions, may need to talk to other humans (all of what you said above)
- humans evaluate correct procedure, double-check it (you don't want them making the wrong "fix" and making something else worse, do you?)
- humans execute commands
- commands take time to run on large collections of computers (running them completely in parallel can cause thundering herd issues, in some cases)
Things change in a time when you would freak getting up in the morning and say google.com did not work.
Automated processes can only mitigate so many edge cases. Even then, humans need to be involved, and that slows things down.
Another example - I can record a video and broadcast internationally, translated on the fly into dozens of languages, effectively communicating with significantly more people than if I could not harness that technical capability. In 2000 that communication process would have been orders of magnitude longer. (Did they have on-the-wire translation then? Idk, just making an assumption to illustrate my point.)