If that is somehow the only barrier between humanity and annihilation, then things don't bode well for us.
Do you mind telling me how exactly you turn that off?
Now we're lucky in the sense we don't have that today. AI still requires data centers inputting massive amounts of power and huge cooling bills. Maybe we'll forever require AI to take stupid large amounts of power. But at the same time, a cray super computer required stupid amounts of power and space, and your cellphone has leaps and bounds more computing power than that.
I can accept that it would be hard to turn off. What I find difficult to accept is that it could exist. What makes you think it could?
Evolution has to take a random walk to get where it is, it doesn't necessarily have a goal past continuation, intelligence shortcuts that. It can apply massively iterative efforts towards a goal.
That doesn’t mean we are anywhere near that level of performance.
You literally have to do nothing, and it will shut off after 24h.
Do you think people would plug them back in if they were working on killing us all?
On another note, I wonder what kind of metric could be used to show the "computing", if that's what is happenning, going on in our brains? This could be interesting to compare to the power consumption of the brain, and then the same thing with a computer, or gpt 4.
I'm fairly certain our brains process orders or orders of magnitude more than any computer running today - but that's all biased towards being a human being, and not pure computation, so much of the processing is "wasted" in running muscles and desciphering audio and visuals.
If it runs on millions of hacked devices, how do you turn it off? The same way any botnet gets turned off: virus scanners clean it up, the C&C servers get taken down, etc. This is not a new problem.
The usual response to this is to claim that superintelligence will develop so fast that we'll go from crappy chatbot to Skynet in 5 seconds flat, and nobody will have a chance to turn it off. That whole scenario is unmoored from reality: it assumes multiple massive leaps in tech that aren't anywhere on the horizon and where it's unclear why anyone would pay for such massive CPU overkill in the first place.
You keep acting like AGI is just going to be some other program. And yes, we do not have AGI yet. No planning, no continuous chain of thinking/learning at this point. But thinking that anti-virus would have a chance in hell again AGI is pretty weak when AV tends to fall apart pretty quick with a human behind the computer. Again, thinking of this as an application and not an adversary would be an incredible mistake. Taking out global adversaries that attack over the internet is near impossible, especially if they have shelter in an enemies foreign country. And this isn't going to be like a human where you kill the leader and it stops. People would be harboring copies for the lulz.
>unclear why anyone would pay for such massive CPU overkill in the first place.
It's also unclear why companies like Google have tens of thousands of engineers at times, but if the application produces useful results, and it's corporate masters think they'll make more profit from it then the operating costs they will gladly keep pouring coal in the furnace for it. And then in military applications one side will make more powerful AI because they fear the other side will make a more powerful AI and get an advantage. I mean we already spend billions of dollars a year upkeeping enough nuclear weapons to flash fry most of the population on earth.
Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere.
Terrorists aren't programs or even a well defined group, so I don't see the analogy.