It's a truly great story, and it obviously shows the author knows their around the topics discussed, but to me the most pleasant surprise was the author's avoidance of humanizing the AI. Most stories about "the singularity" or AI apocalypse in general, immediately presume that just because a machine is self-conscious, it also wants things, or might even feel things. However, machines, even conscious ones, are not people - they do not age, they do not die, they have no biological urges, and unless programmed in particular, have no reason to "feel" fear or threatened by anything. Perhaps AI psychology will one day be a thing, but that will most likely be immensely different than "that is a person in a computer", because it isn't.

It was great to find the author actually acknowledges this: "it cannot ‘want’ anything beyond maximizing a mechanical reward score, which does not come close to capturing the rich flexibility of human desires".

The story certainly makes a strong push towards making you believe that it is possible, especially in trying to explain every little detail, some of which does in fact sound fairly plausible (and some not as much).

To throw a wrench in this grand vision I would point out that anything like this will have a seriously hard time taking off. A sprawling process that gorges too much resources has a high probability of getting killed by the os, especially if it's on an HPC-like machine where a lot of training is taking place. Speaking of which, a lot (if not most) of very serious ML training happens in isolated networks such as corporate or academic VPN, so any AI running on those machines will have a limited reach, even if the AI can somehow magically exploit every Linux kernel it runs across (which by the way, is one of the weaker links in the story IMO).

A somewhat amusing point was when the AI "has already cashed out through other cryptocurrencies and exchanges". The idea of using crypto was interesting, if not wholly necessary for the story. I assume the AI would use a stolen identity since Clippy doesn't have a bank account. Moving a lot of money around, especially from crypto to fiat, is no small feat, and can easily be blocked by the financial system (i.e. banks). Maybe this is the part where "it's set in 20XX" makes it more plausible. Today, we can effectively shut down the entire internet by cutting power to the backbone centers, I wonder if that will still be possible in the future. This of course will be a crippling economic blow, even if the AI doesn't survive.

Overall a great read. If nothing else, it can encourage researchers to isolate their ML ops so that if the singularity happens, you know it didn't come from your own sausage classifier.