The artice is literally about how this will lead to a hostile superhuman intelligence that will wipe out humanity. It's not about AI tooling as an accelerator, which is a valid concern that imo can't be avoided.
An AI we train to take care of us will almost certainly learn to want some other stuff, in addition to not exactly caring about us the way we want. That's fine if it's not too smart, we just stop it from doing dangerous stuff. The problem is when you crank it up to a million, and then those things that are just a tiny bit wrong become a massive problem, and you can't turn off the AI because it is smart enough to understand that the things it wants to do won't get done if it's off, and it doesn't want to let itself be modified in the same way that you wouldn't want someone to give you a pill that makes you stop caring about your children.