Universal function approximators are not about to take over the world.
Universal function approximators are not about to take over the world.
https://www.ianhogarth.com/about/
https://en.m.wikipedia.org/wiki/Ian_Hogarth
Ignoring the unsubstantiated ad hominem for a minute though, it's getting frustrating that the mere introduction of the topic of discussing possible AI futures causes immediate derision.
It's as though the corporate research community, for which I'm associated, is vehemently against even discussing AGI. Whereas Barto, Bengio, Hassabis etc... are happy to discuss it in reasonable ways.
The author seems to be discussing it reasonably and not making crazy kurzweilesque prognostication.
Everyone at the Dartmouth Workshop wanted to create human level intelligence. Let's stop pretending that's not still the goal. OpenAI and DeepMind have that as explicit goals and if you dig into any serious AI researchers they say that's the goal vector.
So where's the beef?
I think it comes from a combination of being scared of overhyping AI and causing the next AI winter, and some form of gatekeeping.
Suppose we have a really strong universal function approximator---stronger than current neural networks, whose generalization properties are not really that great in the grand scheme of things as of 2018. Tell it to approximate the action policy that maximizes some overly-simplistic geopolitical objective function, like GDP or territory controlled at time t+1. It doesn't seem at all obvious that this thing could not take over the world or at least cause significant havoc if given sufficient resources.
The problem is that with such a vague objective as "maximise GDP" or "maximise controlled territory" you need to train a model that is extremely broad in scope - because the objective might be narrow, but the steps to realise it are extremely varied. In practice, you're trying to approximate a function that is outputting the state of the entire world at each time step. Good luck with achieving that in practice.
Edit: The bit about collecting examples is not a trivial problem. Note that the successes of deep learning so far are in domains where not only the objective is well defined ("choose one of n categorical labels") but also the data associated with the objectives is easy to collect and has an obvious relation to the outputs. Say, if you want to train an image classifier to recognise images of dogs- obviously you need to collect images of dogs. This is not the case in "increase GDP" type objectives, where it is not even clear what exactly influences a country's GDP. In principle, you could feed the entire world as examples to the model, but in practice, that's just unfeasible.
You could specify a simpler goal like “increase economic output”, and an example might be something like optimizing the Cobb-Douglas production function. Even that very narrow goal in the context of say, manufacturing Teslas, would give me pause. Look up “instrumental convergence” to see why the above is a bad idea.
From what I can see on wikipedia, the parameters of Cobb-Douglas are the value of goods produced, labour, capital, "total factor productivity" (as I understand it, everything other than labour and capital that might contribute to productivity) and a couple of constants. For an AI to maximise the output of the function you'd have to somehow make it possible for it to manipulate those parameters- to hire or fire personel, to spend or acquire capital and to somehow manage all those unknown factors that might be contributing to the output.
The question is: how do you do that? You can certainly collect, or even auto-generate examples of the inputs and outputs of the functions, since we're just talking numerical parameters, and find a maximum of the function. But for an AI to actually improve the productivity of a business, it would have to do a lot more than that. It would need the ability to manipulate those parameters directly in the real world. Otherwise, all it would do is calculate a number. Which is not that very threatening.
The hard part here is writing down that objective function. Remember, an AI/ML/cogsci algorithm is locked inside a black box, that being the hardware it runs on. Any objective function for RL must be expressed as a function (preferably a smoothly differentiable one, for gradient descent) of the sense-data available to the agent and the agent's hypothesis class about the world. Naive RL tends to optimize the function by, where at all possible, systematically decorrelating the agent's sense-data and reinforcement signal from the distal causes we intend them to represent.
So for example, write down the objective function for current General Intelligence: Humans. It's impossible, and has been the work of the field of Philosophy/Economics since we started seriously thinking about it.
Nothing in that article discusses interpersonal neurological response systems or anything relating to how mores and boundaries are created.
Seems like you're linking to something which may be on the track to narrow down consciousnesses which is a separate question - and one I also question the benefit of caring about.
Well, we weren't discussing social reasoning and behavior, so I linked an article talking about the systems governing the brain's "objective function".
My original statement should have been: "We can't model Humanity's Collective Objective Function" - which is what would be behind what we are interested in: Stable functioning muti-agent systems. I think EY took a crack at this a long time ago and rightly abandoned the concept (see: CEV).
Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs.
Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem.
I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups.
If no such thing exists, then it was the wrong thing to investigate, so stop being interested in it.
>Even with that clarification I disagree with the premise that we can model an "objective function" for an individual strictly in-vivo. Modelling an individual agent's reasoning/function system doesn't account for the environmental context it exists inside of, gives input into and responds to. So even if it was possible to understand the mechanism for intra-personal decision criteria, and I don't think it probably is, I don't think it's generalizable without having the context of inputs.
That's just an inverse reasoning/theory-of-mind problem, one that normal theory-of-mind models and actual human brains solve every day.
>Assuming that we could do this, I don't think you can extrapolate intentionality directly from individual to collective groups - which for an AGI is what is existentially important as it needs to be collectively general to solve the existential problem.
What's this about "collectively general" and "the existential problem"? You seem to have gone off the deep end into philosophy salad.
>I also don't think this is desirable as a framework for AGI - as humans, despite our intelligent status, are quite unstable and sub-optimal in groups.
Considering you don't seem to know much about how humans work and what causes us to work well or badly in various situations, this statement comes off as almost racist.
You broke the HN guidelines pretty badly here, first by getting personal, and second by not doing this: "Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize." The result was a whole litany of complaints elsewhere, which were understandable if off topic (https://news.ycombinator.com/item?id=17359858).
You unfortunately have a long history of breaking the site guidelines. Would you please reread them and fix this? It's really not ok, and even though I appreciate your substantive comments, their value doesn't obviously exceed the damage you've caused with harshness, swipes, and uncharitable responses over the years.
I can only interpret this as a mystical claim...
That's just pure speculation because nobody has a clue what such an intelligence would look like. I can imagine using the mind infinity stone to take over the world, but that doesn't make it realistic.
The trick with AGI is that it can learn faster, and humans (even with our limitations) did take over the world.
Humans are limited by a lot of things like motivation and communication which is likely distinct from the general problem solving architecture itself. Humans also have goals and an evolved morality that guides their thinking. Humans share a similar type of intelligence and understanding of goals, culture, right and wrong. You don't get this for free with AGI.
The point of the human example is that there already exists a general problem solving architecture you can scale up (it's not impossible and there are brains everywhere in nature).
If you increase operations per second many magnitudes you can have all human learning over thousands of years compressed into a couple of hours. You see this loosely in narrow spaces now with things like Alpha Go Zero.
There may not be other AGIs if there's an intelligence explosion and the first one improves itself very quickly. If its utility function is not aligned with human goals then it could end up turning all matter into paperclips - not because it's evil, but because it's hard to set goals tied to human morality (when humans don't even agree in every case) and it happens to be configured such that making paperclips maxes out its reward function by accident.
It may be possible to make the unsafe version before making the safe one, since it seems harder to make an AGI that's aligned with human goals than just an AGI in general.
I think the mismatch comes when people looking from the outside who have no real understanding of how the internals work see this progress in a very narrow domain and extrapolate that idea into thinking that something more general is happening.
When you understand the trick, it stops being magic.
If a machine can generate strategies to win simulated wars, how much harder is it to generate strategies to win real wars?
If machines can out compete people on 50% of jobs, how much harder is it to out compete people on 100% of jobs?
> When you understand the trick, it stops being magic.
The only argument I've seen about the difficulty of general AI is that we don't presently know how to do it. But it only seems difficult (like magic) because of our lack of comprehension. Maybe it actually is, but the good odds are that like everything else it's just one or a couple of tricks, and once we put the pieces together we'll be moving just as fast.
The question is, of course- how? How will AI become able to take over the world?
This is an important question to ask because predictions of AI rapidly becoming capable of taking over the world tend to flare up in times of progress in AI performance. For example, the above article is basically based on the preimse that the latest developments, i.e. deep learning, are sufficient to give some nation-state technological supremacy by leading directly to AGI.
However, there is no clear path from deep learning, however powerful it may be as a set of techniques to train classifiers, to AGI, which (well, probably) entails much more than powerful classification. For instance, deep learning can't really do inference. It seems reasonable to assume that AGI will need the ability to reason about the objects it can recognise and the relations between them. Deep learning can't do causal analysis- like Judea Pearl has been screaming from the rooftops lately. It seems reasonable again to assume that AGI will need to model causal relations. Deep learning can't do semantics. AGI will need to have some handle on semantics. And so on and so forth.
In general, anyone that claims that X current development in AI risks inadvertently producing AGI, must explain how this is supposed to happen- not just wave their hands about and claim that, if you throw enough compute and enough data at the problem, magic will happen and AGI will turn us all into paperclips.