The AI still will be used by a human, unless you think that the actual power moves to the AI and not the humans and their institutions using it.
You just need one person to type "kill every firstborn and tell the parents that this is retribution for the rebellion" instead of that one person convincing thousands of people to follow their (unlawful) order, which might take decades of propaganda and indoctrination. And that one person only sees a clean progress bar instead of having to deal with 18-20 year olds being haunted by nightmares of crying parents and siblings, and baby blood on their uniforms.
Yes, in the end it's a human typing the command, but AI makes it easier than ever for individuals to wield large power unchecked. Most nations make it allowed and many also require soldiers to refuse unlawful orders (i.e. those instructing warcrimes).
And it doesn’t need to even be totally that to be a problem, each incremental bit of progress that reduces the number of people required to apply power over any given size subject population makes tyranny more sustainable.
When your job is to repair and maintain infantry drones, how do you know if it's civilian or combatant blood that you are cleaning from the shell? You never leave the camp because doing so is too dangerous and you also don't know the language anyways. Sure the enemy spreads propaganda that the drones are killing babies but those are most likely just lies...
Or when you are a rocket engineer, there is little difference between your globe spanning satellite network transmitting the position of a hypersonic missile headed for your homeland, or remote controlling a network of drones.
AI gives you a more specialized economy. Instead of every soldier needing to know how to maintain their gun you now have one person responsible for physical maintenance, one person responsible for maintaining the satellite kubernetes network, one UI designer, and one lunatic typing in war crime commands. Everyone in the chain is oblivious of the war crime except for that one person at the end.
Look at what disinformation and astroturf campaigns accomplished without AI in the past ten years. We are in a very uncomfortable position WRT not autonomous AIs but AIs following human-directed agendas.
You can remotely pan a gun around with the right apparatus, and you can remotely pull a trigger with a sister mechanism. Right now we have drone pilots sitting in trailers running machines halfway across the world. Who is to say that some of these human-controlled things havent already been replaced by AI?
I think AI manipulating the physical world is a lot closer than you think.
It isn't obvious that drones are more effective killing people without human pilots (and to the extent they are, that's mostly proneness to accidentally kill people with friendly fire or other errors) and the sort of person that possesses unchecked access to fleets of military drones is going to be pretty powerful when it comes to ordering people to kill other people anyway. And the sort of leader that wants to replace soldiers with drones only his inner circle can exercise control over because he suspects that the drones will be more loyal is the sort of leader that's disproportionately likely to be terminated in a military coup, because military equipment is pretty deadly in human hands too...
Today, already, you can set up a red zone watched by computer vision hooked up to autonomous turrets with orders to fire at anything it recognizes as human.
Some guy already did it for mosquitos.
Sure. You could also achieve broadly similar effects with an autonomous turret with a simple, unintelligent program that fires at anything that moves, or with the 19th century technology of tripwires attached to explosives. The NN wastes less ammo, but it is't a step change in firepower, least of all for someone trying to monopolise power over an entire country.
Dictators have seldom had trouble getting the military on side, and if they can't, then the military has access to at least as much AI and non-AI tech to outgun them. Nobody doubts computers can be used to kill people (lots of things can be used to kill people), it's the idea that computers are some sort of omipotent genie that grants wishes for everyone's firstborn to die being pushed back on here.
China's monstrous social credit thing might already be that.
No consciousness or omnipotence needed.
This is still a "human malice and error" problem, not a "dangerous AI" problem.
Even with AGIs, there is still power in numbers. We still understand how all the mechanical things work. The first attempts of AGIs will be botched anyway.
It's not the AGI will secretly plan for years hiding itself and suddenly launch one final attack.
Also, typical tyrannical leadership doesn't work that way as it tends to get into power being supported by people that get a payoff from it. It would also need a new model of rise to tyranny, so to speak, to get truly singular and independent tyrants.
Yes, that is exactly what I described as the limit case.
But, as I note, the problem exists more generally, even outside of the limit case; the more you concentrate power in a narrow group of humans, the more durable tyranny becomes.
"Give me a lever long enough and a fulcrum on which to place it, and I shall move the world." -- Archimedes