Comment by mkaic | Hacker News Reader

Comment by mkaic | Hacker News Reader

Parent Full thread

Hi! Author here. I think you raise some great points! I'll address them each:

-- I am a professional AI practitioner. I work in the field of medical deep learning and love the field. I am strongly considering starting a few experimental forays into some of the concepts I mentioned in my post as side projects, especially self-modifying model architectures.

-- Yes, you're totally right! I am making the fundamental assumption that it is possible. My reasoning is based on my belief that human behavior is simply a function of the many sensory and environmental inputs we experience. Since neural networks can approximate any function, I believe they can approximate general human behavior.

-- This is fair. The topic of the singularity is often used for traffic/fame (I mean, I'm guilty of this myself with this very post, though I hope I still managed to spark some productive discourse) and so there are always conflicts of interest to take into account. I can't name any examples off the top of my head that perfectly fit your criteria, but depending on how much you trust him, Elon Musk seems to be genuinely concerned about the potential for a malevolent singularity.

Thank you so much for your comment! I really appreciate your feedback. Have a wonderful day.

How can one turn the sinking, horrified feeling when one loses a love into a function? Or describe in terms of a function, the blissful wonder of being in the arms of a lover? An issue I have is the that there seem to be profound limitations to language, explored in the philosophy of language, that fail to capture much, if not most, of the world. Functionalist models of mind and behaviour seem extremely limited, as our subjective ontology doesn't seem to reduce to functional outputs.

You also say that an AI would rapidly consume the whole of human knowledge. For me, the totality of human knowledge would become a mass of contradictory statements, with little to choose between them on a linguistic level.

There are, for me, profound philosophical issues with creating a mind that is "conscious" in the sense that an AGI is implied to be, as a purely symbolic logical construction. Language is the only tool we have for programming a mind, and yet the mind cannot be completely described in language, nor can language seem to properly encompass whatever the fundamental ontology of reality involves. I don't feel there will be a "free lunch" where we advance computer science to the point where we get an explosion of the kind AI1 designs a better AI2, which designs an even better AI3, and so on. This seems to have a perpetual motion feeling to it, rather than one of evolution. It isn't to say AGI is impossible, but I believe that like everything else in computer science it will have to be solved pragmatically, and realising this could be an extremely long way off.

> How can one turn the sinking, horrified feeling when one loses a love into a function?"

The same way the brain does. Those complex feelings can eventually be resolved into dumb neurophysics. Love, fear, anxiety etc. al are just electrical impulses tickling chemicals. Is there anything in our brains that we could never approximate with technology?

This is my view as well. It's a little unnerving and it definitely starts to overlap with the whole "free will" debate, but yeah, I don't see any reason why we can't fundamentally replicate the behaviors exhibited by the brain. It doesn't violate any laws of physics.

We've had technology like poetry, art, and music for thousands of years, and yet no symbolic description of the feeling can contain the feeling of what it's uniquely like for me. Even though we could try and model the brain, say, functionalist models fail to capture qualia as it doesn't reduce to behaviour. To replicate the brain fully in a computer would need a full description of its chemistry and physics, along with that of the greater universe, which we don't have, and to describe it coherently to simulate it is a problem of the order of magnitude more difficult than ones we're going to be able to code for the foreseeable future.

I agree that it'll be while before we fully understand the brain but I don't have any doubts that we'll get there eventually. I am curious though, why would we need to understand the greater universe perfectly as well?

Where does the brain get its inputs from? These computational models are based on an ontology where the brain is an isolated box separate from the universe, which is only one of the many philosophical outlooks argued over. For example, most schools of Buddhist philosophy would regard this separation as entirely the wrong picture of the world.

Philosophers have debated this since at least ancient greece. It's hardly a solved question, and about a third do believe that there is something in our brain that we could never approximate with technology.

Have any of that third put up any testable theories about that something? Or are there leading theories as to what the something might be?

If a philosopher had a testable theory I don't think they'd be a philosopher anymore.

This assumes that there is no free will however :)

> How can one turn the sinking, horrified feeling when one loses a love into a function?

… or into electrical activity in the brain.

Can you point to a paper or website that contains this function, described in full? Yes, electromagnetism no doubt, but while we can postulate a function, we still don't have the function, and will somehow have to write it down for it to be a function.

> somehow have to write it down for it to be a function.

That’s not how AI is trained. You don’t need to know or even understand the resulting model to train it.

The training algorithms have little to do with the resulting model, and are certainly not themselves intelligent.

Why assume that the activity is purely electrical?

It's not purely electrical. It's also chemical, especially if you look at how synapses work. (But then again, chemical processes are driven by electric charges anyway.)

I don't think AGI is seeking to create a machine that can love, and I think it would be even less capable of it than "mere" logical intelligence.

Thank God too, the moral questions involved are truly terrifying (humanity as a Great Demiurge, birthing monstrosities).

> How can one turn the sinking, horrified feeling when one loses a love into a function? Or describe in terms of a function, the blissful wonder of being in the arms of a lover?

Evocative questions but I have to challenge the premise. First, that things like emotions and qualia are design ends in themselves for a successful AGI rather than potential emergent properties of same.

For that matter, are they really necessary to the brief?

> An issue I have is the that there seem to be profound limitations to language, explored in the philosophy of language, that fail to capture much, if not most, of the world.

And how much of the world does a human mind capture?

The piece already accounts for this claim. The theory is that all language has to do is describe a sophisticated enough network. After that it's black boxes all the way down.

> Language is the only tool we have for programming a mind

And a darn good one. Formal languages can express a great deal when you find the right abstractions.

Cubic splines can approximate any function too, so the universality argument is a little weak IMO.

Even if one buys into the idea that human behaviour is a 'function' of sensory and environmental 'inputs', that's a long way from showing a neural net a million different texts and asking it to generalise.

I think the first AGI to pass a Turing test will probably be a simple language model. I don't think it will look like any of the GPTs, but I think text completion is a great starting point. I'm not sure how other inputs will be added into the mix, but I am sure that they will be -- heck, maybe once we train a general language model, it may very well just tell us how to incorporate things like video, audio, haptics, gyro data, etc into its architecture.

^ To be clear, mkaic is the original author of the article.

Oh, whoops, I'll edit the comment to make it more clear. Thanks for reminding me.

Reply on news.ycombinator.com