And good luck holding the military back.
One way or another humanity is on its way to the inevitable, and it's our evolutionary destiny because we are basically hard-wired to pursue the curiosity that is AI.
We can control businesses. And as for someone in their basement, it isn’t clear at all that one could develop AGI in their basement. AGIs discovery will probably require massive, massive amounts of compute. More than could fit in a basement or be powered by domestic grid connections without drawing attention.
Good luck holding the military back? You are talking about the US right? The military and every other government entity answers to the people either directly or indirectly. If enough ordinary people wanted it, the constitution of the United States itself could be amended into the ITunes terms of service. And since AGI is to the detriment of literally every human being on the planet, I think some basic legislation might be able to be passed.
Don’t succumb to these notions of hopelessness. They are wrong. And you even spread them around which just makes solving the problem even harder.
These days there are a lot of high school kids messing around with machine learning. They might not have the theoretical understanding, but I don't think it will require a graduate degree to get there.
I'm not sure why there aren't more hacker tools that apply machine learning? Maybe we're not quite there yet.
We got treaties on nuclear and chemical weapons. It could happen for autonomous weapons and weaponized intelligent agents also. But it might require a serious case of abuse before we get there, as it did for chemical and nuclear...
We would need to consider, however, access to finished products (like trained models). In some cases in the future, having access to the appropriate model might be as dangerous as handing someone a nuke.
How hard would it be for someone to assemble a indiscriminate "killer drone," given the appropriate image recognition and flight control models?
My background is an engineering role in the military and my current path is CS. I don't think it's unfeasible that an individual could build a crude drone that would kill a single person. Building an entire swarm of death robots is a different story.
It is for now. We will get to a point where a negative utilitarian with accesses to genome synthesis technology will be able to kill tens of millions. And this seems like it could become pretty feasible, even from a basement.
I can't help but think the same is true of AI or any technology that could be used for good or bad.
We spend a lot of time trying to control the tools, but less time trying to improve culture and ethics of the operator.
How do you make the majority of the society believe in the same social goals without resorting to tales a-la Christianity during the fall of Roman Empire?
You'll either need to make the whole of the humanity live in the sorts of Star Trek universe, where everyone is equally educated and is equally capable to critical thinking.
Or you end up with the same system where the majority of people are obliged to give their absolute trust to the minority.
In my opinion this is the key question.: How do we get everybody on board with e.g. UN Sustainable Development Goals[1] or something like them?
[1] https://en.wikipedia.org/wiki/Sustainable_Development_Goals
Probably because politicians are more aware of the realities of _current_ human nature rather than some future hypothetical that may never happen at all.
Also there's an important point to be made that the powers that actually owned the majority of nuclear arsenal on the planet had enough missiles to destroy most of the human civilization.
The danger of AI abuse is that existing large, powerful entities will put too much faith in statistical models without understanding their limitations, and will make bad decisions that harm people. Fortunately we have a filtering function -- entities that use these techniques with more nuance will slowly replace the failing ones because they are making better decisions. There will be a long transition as governments desperately try to save the dinosaurs from the consequences of their poor decision making, but if they can't make the transition, they're going to cease being relevant.
AI abuse is just statistical abuse. If a bank has a model based on ten years of real estate data that says, "hey, real estate increases in value, we should leverage into that", then come 2008 they'll find that things that their neat statistical model stated don't work any more. That's all the "dangers of abuse" are.
Maybe you missed the section where he said "Killer drones are a big concern." and "A lot of what is most concerning is not happening in broad daylight. It’s happening in military labs, in security organizations, in private companies providing services to governments or the police."
> it's just a statistical modeling technique
...and gunpowder is just a chemical.