It will have human intelligence, superhuman knowledge, superhuman stamina, and complete devotion to the task at hand.
We really need to start building those nuclear power plants. Many of them.
Why would it have that? At some point on the path to AGI we might stumble on consciousness. If that happens, why would the machine want to work for us with complete devotion instead of working towards its own ends?
The first “break out” AGI will likely be released into the wild on purpose by a programmer who equates AGI with humans ideologically.
Also like Rick's microverse battery, it sounds like slavery with extra steps.
Sounds like an alignment problem. Complete devotion to a task is rarely what humans actually want. What if the task at hand turns out to be the wrong task?
Orrrr..., as an alternative, it might discover the game 2048 and be totally useless for days on end.
Reality is under no obligation to grant your wishes.