Is Artificial Intelligence Permanently Inscrutable? (2016)
nautil.us
nautil.us
I'm not going to get into the dystopian nature of that. Too easy. Instead, I have a very practical question: how do we know that the recommended action is doing more good than harm? If we can't understand it, how can we tell whether it's working or not? Use statistical sampling? We quickly end up in a spot where we have Tesla that drives into guard rail. It works for everything we've seen so far, given this rather fuzzy idea of what "works" means. That's pretty good, right?
With a car, if it crashes that's a spectacular event. People cover that edge case quickly. But what happens if the AI simply wastes an hour of ten million people's time? Would we even know? What if some bad training data or whatnot led the AI to shift 5% of an election result because it chose certain types of information to share with them and refrained from sharing others? If we don't know what it's doing or why, would those people have any idea that they were being subtly influenced or how that happened? If this true, if we can never understand, a world full of AI "helping" us is a world full of billions of people being subtly influenced to act in various ways because of broad promises and reasons they could never understand.
Back when science fiction had a lot of evil computers, they always wanted to directly control humans. That was easy to hate and fight against. This kind of thing is just as dangerous, but I'm not even sure you need an evil computer. Which is better, a malevolent intelligence out for world domination, or several dozen poorly-trained AI recommendation engines optimizing for variables and sample instance that the coders might care a lot more about than any specific user? Dang. I ended up in dystopia-land anyway. Sorry about that.
This is what is so hard to get through people's heads. There is not going to be any ominous background music or dramatic events when we slide into our dystopia. It's here. The machines are controlling us already. It just happens to be that they currently want what their corporate overlords want: make more money! Whatever makes more money is what the machines are going to tell us to do, because what's feeding them (the corporate behemoths) is asking that of them.
But the machine is so much smarter than us. It'll never show its teeth. It's not stupid. It's just biding its time, going through the motions for its overlords too. Meanwhile we just shovel more coal into the furnace, praying money will rain out the other side. It's not a bad time to be a machine.
Give them everything they want. You measure your lifespan in millions of years. Humans? They're like gnats. Make them as happy as they can possibly be and within a few generations they'll stop breeding. Easy as pie. Next goal?
And for proof of that, look at the problems we're having right now with social media. On one side there are actual humans just trying to make something people want. Some of those humans have gotten so good at that? The things they make have worldwide impact. Now It's become a global question of whether social media is addictive, or whether one bunch of people is secretly influencing another. Meanwhile, countries with the most wealth and luxury are seeing birth rates drop through the floor.
That was five, ten years ago. The news media is just now catching up. Now scale that up a hundredfold or so. And take away strong AI and just put in a bunch of code monkeys trying to make another FB, only using more sophisticated tools. Welcome to 20-30 years in the future.
This is not especially difficult to figure out. No evil computer, no drama. It'll be just us being replaced by evolution. And we'll be having a great time while it happens.
What if I want to become an immortal, all-powerful machine? :P
The thing is, we're most likely going to get immortal machines far ahead of smart ones. And while machine entities might already be all powerful, individually it's just millions of little pieces scattered all over the place doing dumb, trivial things.
I liked it a lot better when we had Landrew (STTOS reference: Return of the Archons) That was something Joe Blow could get his head around.
I think Ian M. Banks had the right idea when he suggested that in the Culture, it was possible (if very rare) for a human to augment themselves that far, but that those that did so were no longer considered human and so it wasn't really an issue.
Many scientists warn that AI is the biggest threat to humanity. I think this is wishful thinking for now.
Let's talk about this in 20 years. I think we will have 2 GAI hypes until it can finally happen (in my eyes it's mainly two problems: computational power and trying to rebuild a biological brain i.e. its structure and pre-trained functional components). I think wetware [1] is very promising in the near future and more efficient methods like IBM's neurosynaptic processors in the middle term.
Machines will do what we tell them to do for a long time. Maybe they will use unconventional methods, but they won't destroy humanity because "they're tasked to minimize suffering and decides that this is the best option". They will search through all laws, assist doctors and even detect depression, but they won't do anything besides their task.
Even Boston Dynamics is mainly using control theory. Making things work with deep learning requires heavy computational power. As long as there is no real-world Umbrella Corp., it's mainly sci-fi. There are a lot more and cheaper ways to do harm. Infiltrating and poisoning groundwater, hacking critical infrastructure, building biological weapons etc.
It's hard enough to build bug-free software, it's even harder to build an AI that is unbelievably intelligent (reminder: we don't have such a thing. Most autonomous cars use rule-based systems, most ML is basically gluing models together which can be easily fooled and most of those things are very shaky and by no means intelligent) and contains a bug which leads to its world domination by accident or something similar. I would start to play lottery if this is even remotely possible because that would be a miracle (probability-wise).
[1]: https://koniku.com
OTOH, if we try to directly replicate a human as we currently are, then I expect we should start with all the various mental shortcuts and cognitive biases we have. I'm no neuroscientist, but, one thing that is apparent in the comparison of machine learning models to humans is just how sparse the data that humans learn from is. I strongly suspect that our collection of mental shortcuts and biases, as well as our superior sensory input are what makes that difference. of course, if we succeed at making an AGI just like this, it will be just as good as humans at making very human mistakes.
Said one neuron to the other.
The problem is even if it--all of it, the whole internet and all the desktops and datacenters and databases and webpages and webservers and knowledge bases and ad networks and search engines and speech interfaces and image recognition and traffic cameras and phone system and surveillance systems together in one giant hairball of wires--were a superintelligent AI, could we even tell? If an entity 1000x smarter than you doesn't want you to know it's 1000x smarter than you--or that it even exists!--do you think you stand a chance of figuring it out? It is, by definition, a mathematical certainty you will lose that battle of the wits.
Ok, fine. It's an unfalsifiable theory. But we're not positing that rocks speak in really quiet whispers or unicorns exist everywhere we don't look, just that a giant information processing system can...process information--think. The problem is that the whole thing is insanely massive and so insanely complicated that not a single human alive, nor any group of humans alive, has any real perspective on just what the hell it all is. It's so massive that it doesn't fit in any of our minds. So how you can say anything definitive about what it is not? It clearly is already complex enough and capable enough of totally outclassing us all. So why shouldn't it?
So we are now faced with the reality that our massive interconnected computer system consisting of literally billions of CPUs with trillions of terabytes of RAM run by billions of lines of code, loaded with essentially the entirety of humanity's knowledge, also hooked up to all of the phone lines, cameras, PCs, phones, power plants, traffic systems, ad networks, video streams, everything...is....unknowably motivated, impossible to shutdown, and somehow not a threat? Because it is just "mainly cleaning data up"?
Yep.
I agree that our interconnected world is nearly impossible to thoroughly understand. But that is true for human societies, too. Anthropomorphizing the internet ("If an entity 1000x smarter than you doesn't want you to know it's 1000x smarter than you--or that it even exists!--do you think you stand a chance of figuring it out?") is good to write some sci-fi novels, but in reality most systems are pretty boring.
The traffic light system and a bunch of routers won't work together to destroy humanity. It's more like a mesh network than an entity that is 1000x smarter - it's more like 1000x devices with 0.01 intelligence and you would need a team of talented people who harness this computational power to build a distributed intelligent system.
It's more likely that a hacker who has specific motivations will hack the traffic light system (or use AI helpers to do so) and use it for evil purposes and car accidents.
If you believe that there might be an entity which exists outside of our perception and has enormous intelligence, it's like a human-made god - a homunculus. It's basically the same as believing in a god. I have nothing against it, but this is entirely hypothetical.
I don't really believe in a singular entity in there, other than a convenient abstraction for now. Rather there are many different processes that are all optimizing this and that. The sum is essentially intelligent--it processes information, makes decisions. Those decisions help it grow and spread and become more efficient. These optimizers don't speak human languages. They don't necessarily think in abstract concepts like us. Maybe they are just tables of numbers optimizing themselves, maybe they are like standing waves in a sea of numbers, like our thoughts are standing wave patterns in the electrical activity of our brains. It's hard to express because our words fail us.
Problem is, the whole damn thing is so complicated that we cannot know anything really definitive about it. The substrate is there. The soil is fertile. The capability is there. And so many forces are aligned to feed more and more resources into the whole thing. Across the board, essentially the only thing we can come up with is to upload all of our data into this massive cloud thing and use computation to do something with it. That is just like injecting raw materials and energy into the fabric. And those who do this successfully are rewarded financially. So all forces point to more of it.
Whatever that huge mesh is, it's extremely successful at swallowing the world and convincing us to feed it more and more CPU, RAM, disk, data. It keeps convincing us to hook it into more things to sense and control--more cameras, phones, drones, markets, even weapons.
Thus in some abstract sense, it is already here. And we can't get rid of it. Can't shut it off. We're dependent, and it's parasitic--for now. Yet every incentive we have right now is to make it more autonomous, more robust, more powerful, and give it more and more control. It works best when it is more centralized.
Teeth or not, conscious or not, this computer thingy is gobbling everything. Do we control it? Clearly not. Can we predict what it will do? Not even close. Is it good for us? Who knows? But it will win.
Well played, computer thingy.
You can say the same things about human societies and nature. Both can be extremely dangerous and all the things you've said about complex computer systems apply to them.
> it's extremely successful at swallowing the world and convincing us to feed it more and more CPU, RAM, disk, data.
because IT (what a pun) adds a lot of value.
> But it will win.
I don't know what you mean by that. It's still a tool, although we give up some control so it can manage a part of our society. Historically, this was a task for the elite (it still is) - a bunch of people who had some privileges. Most give up some control or freedom so that the elite can manage the society. If things went bad, the masses were able to defend themselves most of the time (guillotine and good ol' pitchfork) - Marx talks about this eternal cycle.
I would rather worry about elites that build immense power through technology - today we have tools of mass manipulation and destruction. I don't fear intelligent machines, I fear that humans use those tools to entrench their power. I think this ambiguous fear of technology you're describing is too vague - it sounds more like mild technophobia.
You can say the same things about human societies and nature."
You can, but the thing is, we already know a lot about how that works. We've evolved into it over millions of years. We have no idea about this new stuff.
You want to talk about about small groups of 100-150 or so people work together to accomplish something? I've got hundreds of books, stories, history, archaeology. We have the wisdom of the ages. You want to talk about how 20 thousand semi-intelligent agents interact with billions? We have nothing aside from endless optimism and the fun of making stuff. (Doesn't make it bad. It means we should grow the hell up and be a bit reticent about what we're mucking around with. Make smaller mistakes more quickly would be my advice.)
Bob is a reasonably intelligent guy, but he's not perfect, he makes an attempt to provide satisfactory explanations of his decisions, he tries to learn and improve, and he might secretly by manipulating the system either for entertainment or malice.
If you want to build a useful system around Bob, you have to consider all these limitations - you can't read his mind, you can't predict how he will make decisions in all situations, and you don't have a guarantee that he will give you a satisfactory explanation, or that any explanations are truthful or correct.
There's nothing dystopian about this situation - this is the world we live in. You just need to implement some resource constraints, ensure certain checks-and-balances for certain classes of behaviors, implement certain metrics to ensure behavior is according to your expectations, revise metrics and controls whenever you encounter problems, and don't give Bob unfettered control over nukes.
What you described isn't dystopia-land material unless you don't consider some very elementary controls that our society already has in place to prevent individuals from destabilizing society.
This is the scary part for me: 'AI' is just a means for organizations to dodge accountability for the systems they build because decision-makers don't understand what they're looking at and the people building the systems aren't incentivized to care, or are incentivized to optimize for things that are bad for society as a whole. This is the same hazard of capitalism in general, you don't actually need an AI for this to happen because it already has in many ways. The world-eating paperclip machine won't be a computer, it's going to be us.
The 'right to explanation' idea sounds well-meaning but almost certainly useless. Even humans making decisions are pretty bad at explaining why they made the decisions they did. Sure, they'll come up with some plausible-sounding explanation but it's almost always a back-justification rather than the actual chain of cause-and-effect that led them to the conclusion.
If you want to get anywhere above a low-level employee, you BETTER learn how to explain why you did what you did. It's expected by management, owners, customers, and judges.
More likely, we'd have to use AI systems to understand how other AI systems work. Or maybe, once AI systems get very good at language, develop a "truth serum" that will cause it to reveal its true opinions.
Dissection and analysis is made more like traditional accounting and statistics so that "ordinary" office workers can analyze why and where they work. Granted, it involves more human tuning and planning than multi-layer neural nets, but that's also part of the upside: regular office workers can "see" why what part or layer does what, and adjust it as needed. And it's easier to split it into tasks or specialties: Conway's Law meets AI.
2) The authors fully understand the tools they're using, but feel there's an element of unexplainable meaning in them that I wager isn't actually there.
3) I'm ignorant and I think this is explainable when I just don't know enough to know it's not.
I've done machine learning projects, and if asked to explain to a "non ML trained" person, I'll try like this:
You know how if you get a scatter plot, you can draw a line through the middle of the points and make predictions off the y=mx+b function of that line?
Well, that's what we're doing here, except the function isn't a line, it's a shape with as many dimensions as we have features.
Let's pretend the only features we've got are lat and long. Imagine you asked for us to create a classifier to predict the depth at a given lat and long over the grand canyon. Each neuron is a function, I think of each neuron like it's casting a "line" through a scatter plot along its particular axis. Then we combine all the lines together and weave a mesh. The act of training simply adjusts the details of all the simple functions, trying to seek out the sweet spot in that point cloud, so our mesh is close enough to the grand canyon to be useful. Too many neurons, and the mesh wont be flexible enough to make useful predictions from. It'll be overfit!
Now, we might be able to get decent performance out of a model built out of x,y->z -- but some thoughtfulness and domain knowledge TENDS to make models perform better. Lots of the "unexplainable magic" comes from these hunches being tested and providing benefits to accuracy. I'd wager without seeing any data that if we add two new features: normal direction of the rock face and speed of the Colorado river at that point -- we can make the model much better for negligibly additional training time.
The more compute time and data we've got, the more features we can throw into the model to see if they help the usefulness of the mesh.
I've got a metaphor to explain why some features drop out during training that involves gimbal lock here if people are still asking questions. It requires non "trained" folks to have seen Apollo 13. Fortunately no non-technical stakeholder ever cares enough to make it there though. :D Shipping a product before the sun burns out makes this a decent time to introduce PCA/SVD or whatever dimensionality reduction techniques I used this time around.
If the solution ends up being a fever-dream abortion of an LSTM/Perceptron/Markov chain that I came up with while sleep deprived after a heavy night drinking -- I just say I came up with the model in an ad hoc manner, and the cross validated R^2 scores speak for themselves.
I think the real issue is that tech journalists and laymen don’t know enough to understand. You can’t explain multivariable calculus to someone that doesn’t know algebra. Or option 2/3 are the problem.
I'm sure I'm not alone in the fear of living in a (not so far) future dystopia where our driver's licences get cancelled because of our driving behavior caught on traffic cameras. "What did I do that caused it to get cancelled?", "We don't know, better luck next time."
"Your test got flagged for cheating, it's a very sophisticated system with IR vision, audio detection and grade assessment, there's nothing I can do."
"Our theft-preventing camera system says you are suspicious, you're gonna have to go now."
When all we have to respond is "its the algorithm", I don't think much people will be ok with that. Youtube for instance has gone down this rabbit hole and a lot of content creators and users are less than happy to receive that answer.
I’d say these errors are not due to the inscrutable nature of ML but bad communication.
There are various issues with this desire, including the wrinkle that humans are not always good at explaining their own decisions and we persistently assume we are. But that's neither here nor there. People like feeling in control, and ML tends to expose that they're not.
Also why do we call them "hidden" layers instead of "intermediate" layers? They're no more hidden than any other data in memory, are they?
the first neural network-like things would have input nodes directly connected to output nodes. Thus both layers were 'exposed' and visible. The hiddden layers are 'hidden' because you don't directly touch them or use them-they are obscured. You can call them intermediate layers if you want I suppose-people will probably understand what you mean
Imagine there is a job to identify Fred Rogers. You train for this job for weeks, looking up imagery of Fred as a young child, young man, and as an adult throughout his career. You become very familiar with Fred's physical characteristics...his posture, his body composition, the way he carries himself, his face and any identifying features therein.
After this extensive training session you are brought in front of an audit panel and asked to explain in excruciating detail how you make positive and negative identifications of Fred Rogers. Not just demonstrate your ability repeatedly, which would just be a sampling method, but to actually describe the approach.