Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.
Edit: it could still be true though. I guess I like some more handholding and pre-chewing before giving governments and large corporations more ropes.
Good argument, lets ignore the (human) elephant in the room!
>worrying about deadly quantum laser weapons
If humans were shooting smaller less deadly quantum lasers out of their eyes I'd be very fucking worried that we'd make a much more powerful artificial version.
Tell me why do you think humans are the pinnacle of intelligence? What was the evolutionary requirement that somehow pushed us to this level?
You simply cannot answer that last question. Humans have a tiny power budget. We have a fit out of the birth canal limitation that cuts down on our brain size. We have a "don't starve to death" evolutionary pressure that was the biggest culling factor of all up to about 150 years ago. The idea that we couldn't build a better intelligence optimized system than nature is foreign to me, nature simply was not trying to achieve that goal.
If that is somehow the only barrier between humanity and annihilation, then things don't bode well for us.
Do you mind telling me how exactly you turn that off?
Now we're lucky in the sense we don't have that today. AI still requires data centers inputting massive amounts of power and huge cooling bills. Maybe we'll forever require AI to take stupid large amounts of power. But at the same time, a cray super computer required stupid amounts of power and space, and your cellphone has leaps and bounds more computing power than that.
I can accept that it would be hard to turn off. What I find difficult to accept is that it could exist. What makes you think it could?
Evolution has to take a random walk to get where it is, it doesn't necessarily have a goal past continuation, intelligence shortcuts that. It can apply massively iterative efforts towards a goal.
That doesn’t mean we are anywhere near that level of performance.
You literally have to do nothing, and it will shut off after 24h.
Do you think people would plug them back in if they were working on killing us all?
On another note, I wonder what kind of metric could be used to show the "computing", if that's what is happenning, going on in our brains? This could be interesting to compare to the power consumption of the brain, and then the same thing with a computer, or gpt 4.
I'm fairly certain our brains process orders or orders of magnitude more than any computer running today - but that's all biased towards being a human being, and not pure computation, so much of the processing is "wasted" in running muscles and desciphering audio and visuals.
If it runs on millions of hacked devices, how do you turn it off? The same way any botnet gets turned off: virus scanners clean it up, the C&C servers get taken down, etc. This is not a new problem.
The usual response to this is to claim that superintelligence will develop so fast that we'll go from crappy chatbot to Skynet in 5 seconds flat, and nobody will have a chance to turn it off. That whole scenario is unmoored from reality: it assumes multiple massive leaps in tech that aren't anywhere on the horizon and where it's unclear why anyone would pay for such massive CPU overkill in the first place.
You keep acting like AGI is just going to be some other program. And yes, we do not have AGI yet. No planning, no continuous chain of thinking/learning at this point. But thinking that anti-virus would have a chance in hell again AGI is pretty weak when AV tends to fall apart pretty quick with a human behind the computer. Again, thinking of this as an application and not an adversary would be an incredible mistake. Taking out global adversaries that attack over the internet is near impossible, especially if they have shelter in an enemies foreign country. And this isn't going to be like a human where you kill the leader and it stops. People would be harboring copies for the lulz.
>unclear why anyone would pay for such massive CPU overkill in the first place.
It's also unclear why companies like Google have tens of thousands of engineers at times, but if the application produces useful results, and it's corporate masters think they'll make more profit from it then the operating costs they will gladly keep pouring coal in the furnace for it. And then in military applications one side will make more powerful AI because they fear the other side will make a more powerful AI and get an advantage. I mean we already spend billions of dollars a year upkeeping enough nuclear weapons to flash fry most of the population on earth.
Yes, you can assert that an AGI would be a superhuman level hacker and so no rules would apply. It gets back to the same place all discussions about AGI risk get to - a religious argument that the entity in question would be omniscient and omnipotent therefore it can do anything. Not interested in such discussions, they lead nowhere.
Terrorists aren't programs or even a well defined group, so I don't see the analogy.
We don’t know it’s possible to build superintelligences but so far we don’t have a good reason to think we can’t and we have complete certainty that humans will spend immense, immense resources getting as close as they can as fast as they can.
Very different from the lasers.
You don’t have to build it, just show how it could be super intelligent.
GPT shows that there's enough "there" to draw billions of dollars and thousands of the smartest researchers on the planet into the general problem space of creating intelligent machines. I suppose the closer analog would be to Szilard's chain reaction work which prompted the Manhattan Project.
There might be another AI winter or 4 or 10 or 100 before superintelligence, but they will almost certainly be winters and not ends of the road.
If there is any route to superintelligence and we don't annihilate ourselves by some other method first, we will get there. I don't get what the conceptual challenge with superintelligence is. Do you believe that the brain that happens to exist inside the naked apes of Earth is the absolute pinnacle of possible intelligence?
Well, first of all I do not believe that, but now you mentioned it I assume you do not believe we are the pinnacle of intelligence. Any proof? Because anybody can present literally all of Earth's life as proof that we are.
It is completely unknown to us if and how any great improvements are possible. This quest shares some aspect of the quest for extraterrestial intelligence in that way. It should be out there, because it's hard to think of reasons it can't be, but yet the skies are empty. They could be on our doorstep or the universe could be devoid of intelligent life and anything in between. There's literally no way to tell at this point.
I guess what I'm saying is that it's a bit early to starting beating drums.
Directed energy weapons will almost certainly exist eventually, to some extent they already do.
The reason why it makes more sense to worry about AGI than laser weapons is that when you try to make a laser weapon but fail slightly not much happens: either you miss the target or it doesn't fire.
When you try to make an aligned superintelligence and slightly fail you potentially end up with an unaligned superintelligence, hence the panic.
Lots of tech is like that, but it turns out we don’t get anything done if we keep fearmongering.