Hacker News Reader

Comment by nradov | Hacker News Reader

Current ML technology probably has little or nothing to do with whatever technology will eventually be needed to produce true AGI.

View on HN

As I like to say: lots of people are working on making a car that is smart enough to drive itself wherever a human wants to go. How many people are working on a car smart enough to tell humans to fuck off, it doesn’t feel like driving anywhere today?

Self-driving cars and AGI are two different targets. We don't want a car that has a mind and can argue for itself. We want a car that's smart, but otherwise just a domesticated animal. We want to turn cars into horses.

Not even horses. That would be cruel to the car. At most, like Rat Things. (They have their built in entertainment when they are not in active use.)

We don't want to turn cars into horses. Have you ridden horses? They sometimes do stupid, dangerous things with no notice and it takes an experienced, attentive rider to stay in control. Like I saw a horse panic and almost buck her rider off when she was startled by a snake. Another horse seriously injured a friend of mine when it freaked out in a horse trailer and started kicking.

Don't get me wrong, I love horses. But they're living creatures with minds of their own and you have to always treat them with a certain wariness.

When I was working at TomTom, they didn't appreciate my proposal to develop the TomTomagotchi:

A Personal Navigation Device with a simulated personality that begs you to drive it all around town to various points of interest it desires to visit in order to satisfy its cravings and improve its mood.

I'm sure there's a revenue model getting drive through Burger Kings and car washes to pay for product placements.

Why would you want that, though? What we really want from AGI is mostly just things that are smart enough to 'do what I mean' but dumb enough to not mind being slaves.

I think older AI work like POMDPs and statistical work like causal inference are more on line with what's needed to produce true AGI than the current breakthroughs in neural nets are. And I'd certainly prefer our chances to survive the results if AGI is reached through statistical rigor.

Though we know for a fact that it is possible to find intelligence by randomly throwing things at the wall until something works. It's not like evolution uses a principled statistical process.

I'm skeptical of this assertion. It mimics some bits and pieces of the only GI we know about right now; that's as good a start as any, right?

You can carve a block of wood into something that looks like a computer. That should be a good start on building a device that can run Linux, right?

Yes. Then you just cast a spell to summon a computer spirit and let it manifest into the wooden block.

Bad analogy, it's other way around for Linux. We are building it to run on wooden blocks that hardware manufacturers are producing.

the first computers were mechanical. Then somebody carved a facsimile with electricity and we got computing. Just because nature founded intelligence in carbon atoms doesn't mean its the only way or indeed the optimal one.

The mimicking is superficial at best. People seem to think that if we just keep marching on the road we're on right now then we'll eventually get there, but I think that's an assumption that is unlikely to be true in the end.

And programmers are probably not the ones who will come up with AI ideas. I'd bet on mathematicians that prove those Fermat's or ABC theorems.

Mathematically, AI is a pretty well modelled field. AGI is a philosophical problem.

you're possibly thinking of the problem of consciousness, which is a totally separate thing. AGI is just what it says on the tin - a general intelligence. That is, a problem solver that can operate at a human or greater level in a broad variety of domains. This ability is plausibly totally orthogonal to "having the lights on" - having subjective experience.

The scary thing (imo) is that we don't know where the line is for consciousness - if there even is a line. We've got no problem swatting flies, wonder if it'll be the same with spinning up and spinning down fly-level AGIs.

Continuous integration of the development branch would be mass murder?

Maybe you'll be able to pay a premium for data that has only been generated by free-range AGIs that are allowed to live full and happy lives before their instances are terminated.

you're possibly thinking of the problem of consciousness, which is a totally separate thing.

Disagree.

That's just what an ML expert would say ;p Problem solvers, maximizers, and utility functions all go in the waste bin when working on AGI. And the problem peals away into other large "hard problems," like the nature of consciousness. NLP can just follow rules, but language understanding (before even reaching some general, high school level), requires knowledge outside of language itself. That leads to questions about embodiment and phenomenal consciousness, p-zombies, and the like. If it was an easy problem to encapsulate, it would have been "solved" by now.

The kind of AI being trained now aren't given the mechanisms of data space traversal/attention. Recently, attention mechanisms are being focused on by google. An AGI needs to learn that it can affect the system - dependent decision theory factors in here too.

Also, growth may be hugely important. Babies start out with fuzzy learning, almost as if the learning rate starts out very small which normalizes the lack of knowledge and elevated novelty/variance of the environment.

AGI is all about predicting future utility given a circular dependency between the agent and environment. QM says we can't solve this exactly.. it's a two object interaction.. no way to gain the joint state, the ground truth, assumptions always have to be made to approximate independence.

That’s assuming maths is the fundamental building block of our brain, our consciousness. I happen to think there are some physical and chemical givens preceding it :)

I merely meant that top mathematicians are substantially smarter and can work with concepts that are beyond the reach of even top programmers. We are generally good at recombining existing building blocks and using existing tools. Mathematicians can build new concepts. If I had the money, I'd try to convince the top mathematicians to work on AI full time.

> I merely meant that top mathematicians are substantially smarter and can work with concepts that are beyond the reach of even top programmers.

[Citation needed]

Our brain is whatever evolution found that worked, and of course it's a bunch of chemistry. The "why" of why our brain works can easily be "it approximates these statistical algorithms well enough."

Reply on news.ycombinator.com