Hacker News Reader

Comment by jqpabc123 | Hacker News Reader

I know for a fact that consciousness is an emergent property of binary logic.

Just keep writing bigger and more complex software and one day --- pooff! By some magic neither planned nor understood, the binary logic playback device (aka a "computer") will jump the fence at the reservation for inanimate objects and start writing it's own software.

I know it's true because people make movies about it --- and lots of people on the internet *believe* this is how it works. If enough people *believe*, over time it becomes real. No logic or proof required, this is just how religions work.

A hammer *can* drive it's own nails --- you just have to believe hard enough.

View on HN

Don't get me wrong, I recognise the snark, but GPT-3 already writes code if you ask nicely.

Biggest problem in this area, is nobody really knows what "consciousness" means (or if they are completely certain, then they also disagree with someone else who is also completely certain and/or deny the logical consequences of that definition).

GPT-3 already writes code if you ask nicely

And yet, both the binary logic software and the playback device following the script are completely unaware of what they are doing or why. In terms of consciousness, they are as dumb as a hammer.

In other words, any "consciousness" in this is strictly limited to the author of the software and those who interpret the results. Otherwise, on it's own, it has no impact on reality. An ant comes way closer to exhibiting self awareness and consciousness.

Would you like to have a go at defining "consciousness"? ;)

Sure --- independent thought and action, self awareness and motivation --- all the things that inanimate objects lack and will/have never aquired.

> independent thought

What is a thought, if not a pattern of information that in humans is embodied in the energy flowing though our neurons, and which finds an analogy in the information flow though an artificial neural network?

Are human thoughts really any more independent than any self-learning process that happens to be implemented in silicon? What makes the tiny warm bags of chemicals that are our cells, special?

> self awareness

Can you define this in a way that it simultaneously (1) is something that humans do actually have, and (2) is not trivially accomplished by giving an AI access to its own state, whether by internal feedback loops in a neutral net, by embodiment in a real or simulated environment where it can sense its own embodiment as another agent in that environment, or by other similar means?

(Edit: also simultaneously with (3), no references to the word "consciousness")

> motivation

Internal states within an agent which create a disposition to engage in specific actions?

I definitely don't need much to pass that test: both (original 1993) Doom and bacterial cells demonstrate this.

Are human thoughts really any more independent than any self-learning process that happens to be implemented in silicon?

Yes. It's called "creativity" and "independent thought" --- not programming.

What makes the tiny warm bags of chemicals that are our cells, special?

Maybe quantum physics? This remains to be fully answered but their "specialness" compared to silicon/sand is not in doubt.

> It's called "creativity" and "independent thought" --- not programming.

The AI under discussion here are no more programmed than you or me. The architecture of our brains comes from our DNA, an artificial neutral net from hyperparameters, but in both cases the interesting stuff is learned. The billions of parameters in GPT-3 and DALL•E 2 etc. are beyond the means of humans to program.

Edit:

> Maybe quantum physics? This remains to be answered but their "specialness" compared to silicon is not in doubt.

Quantum computers exist. They are no more animate than classical computers. Would you expect to change your mind if you saw an AI running on a quantum computer, or would you expect to continue to insist that brains are special?

Edit edit:

Can you define "creativity" without reference to any of the other things you're trying to use to define consciousness?

> all the things that inanimate objects lack and will/have never aquired.

Ah, the standard question-begging ontology backed by nothing more than arguments from incredulity.

I submit that a roomba set to an automatic schedule is conscious under your definition.

I submit there is nothing independent about what a roomba does. It is following a script just like very other piece of software ever written. It doesn't know or care that the result cleans your floor.

Our bodies implement a script, which is our genome. We didn't know or care about that code for the first (million?) years [0], and most of us still don't.

Indeed, this lack of awareness of (and lack of regard for) the lower level of our nature, is why we were very happy to invent birth control, and why the people who get upset about the existence of it are seen by the rest of us as a bit odd.

[0] Depends what you count as a human/person/creature with consciousness, which is of course the point of this thread. Yay for blurry lines!

Our bodies implement a script, which is our genome.

Our genome is not a script --- more like a blueprint.

Once construction is finished, the thoughts and actions of your mind/body are not "scripted" --- unless you *believe* in pre-determination.

That also describes GPT-3. The hyperperameters are a blueprint.

If you don't want to count the data flow in a human brain as "scripted", you can't count the data flow in an artificial neural network as scripted; if you do count the ANN as scripted, then you should also count brains as such.

Current AI are, it is worth pointing out, less complex than a human brain: GPT-3 is appropriately the same number of parameters as a rat has synapses, while DALL•E 2 is approximately a honey bee. But the argument you're making has not yet touched scale.

That also describes GPT-3. The hyperperameters are a blueprint.

Can it diagnose my car engine? It seems to be idling rough and it's using too much gas. Does the throttle body need cleaning or maybe one of the coil packs is failing?

> Can it diagnose my car engine?

Probably.

My free credit has run out and I've not gotten around to setting up a payment method for more credit, but this kind of random question is the kind of thing it's surprisingly good at.

Can it really *diagnose* it --- or will it just put together a technically correct and properly structured reply that may sound kinda plausible to the uninformed but which lacks any real insight?

Does it ever object to answering a question? What does it think about in it's spare time? Is it scared someone will turn it off?

The distinction here tends to highlight the difference between consciousness and programming.

Too late to edit the other content, but I think this is a perfect example of humans just putting "together a technically correct and properly structured reply that may sound kinda plausible to the uninformed but which lacks any real insight" when asked to draw a bicycle from memory:

https://www.mensjournal.com/adventure/people-asked-to-draw-b...

> will it just put together a technically correct and properly structured reply that may sound kinda plausible to the uninformed but which lacks any real insight?

Most humans would fail this standard. (Hence the memes "Never read the comments!" and "Eternal September" and widespread beliefs in paranormal and so on).

The (surprising to me) difference is that GPT-3 generally does a bit better than, e.g. Hollywood script writers when facing similar prompts.

This surprising quality is why GPT-3 led to GitHub Copilot.

I certainly hope to never operate independent of my brain and its script! That would be quite worrying.

Would the Roomba be conscious if it had a dust sensor? Then it would know that the result cleans the floor.

I certainly hope to never operate independent of my brain and its script!

Are you a robot? Speaking strictly for myself, my brain does not follow a script. I normally operate on my own --- like when I formulated this reply to you.

I am indeed a meat robot.

I think this is at risk of becoming a debate over the word "script". Is it a matter of complexity? Is GPT-3 a script? Is Windows a script, even though it has more bytes of behavioral code than the GPT-3 weights? Will the Roomba be conscious if I replace its programming with a neural network, even if it behaves the same way? What if I add code to randomly vary its behavior? What if I add code to learn at runtime, considering that the learning code is still perfectly static?

I am indeed a meat robot.

You have my sympathy.

Next, we may expand the length of this thread threefold by asking you to describe free will, and what it would take under the definition of free will you pick (none of which are very satisfying) for a bio system to not have free will, or for a machine to have it

What's consciousness have to do with impact on reality?

It's just one of the things that inanimate objects can't do --- they can't alter their surroundings --- certainly not "consciously".

Two problems there:

(1) "Animate vs. inanimate" is a separate axis to «every defining feature you've just used for consciousness» vs. «the absence of that»

(2) Inanimate AI can trivially alter their surroundings, all I have to do is say "Alexa, Badezimmer zehn Prozent". And this isn't an AI that I hold in high regard or consider to be conscious or anything like that — I have to ask three times while getting increasingly annoyed, because for some reason Alexa can't hear me properly and the first response is usually "Ich kann nicht Badezimmer auf Spotify finden" even though we don't have Spotify.

Still alters it's surroundings, though.

So the sun is conscious?

No but fire is --- it's always mean and destructive.

Inanimate objects can certainly alter their surroundings, or life could never have arisen.

Are you sure they did this all by themselves --- no external forces and/or quantum physics involved?

Why are those suddenly important? Silicon computers involve both, even when un-networked and alone in a cupboard.

A lightening strike nearby once caused by DVD player to turn on. Does that make it "conscious"?

According to your definition:

> It's just one of the things that inanimate objects can't do --- they can't alter their surroundings

And given that you wrote that as a response to the question:

> What's consciousness have to do with impact on reality?

Then you must logically say that it is.

Fortunately I don't have to claim such a belief for myself, because I said earlier:

> nobody really knows what "consciousness" means (or if they are completely certain, then they also disagree with someone else who is also completely certain and/or deny the logical consequences of that definition).

With specific emphasis on "deny the logical consequences of that definition".

Ah well. Perhaps one day, we'll have an AI that can tell us what consciousness is without repeating all the mistakes and pitfalls that humans make when trying to philosophise about this topic.

The DVD player didn't *do* anything. It was all the result of the lightning strike.

If you wish to make that claim now, then you appear to be mixing your definitions in the middle of your argument.

Yes

I don't know if anyone disagrees with the following definition:

Consciousness is the perception of self, which is the result of an information model on self built by the brain from a) the external experiences of the entity, b) the internal experiences of the entity.

Personally I find the above definition 'good enough' for all purposes. Are there any disagreements on the above?

The problem is not whether or not any definition of consciousness is correct, the problem that even if it can be said to be correct, such definitions often just shift the undefinedness and hairiness of consciousness to some other term.

In your specific case, for example, there are non-trivial issues with the words "perception", "self", and "internal experiences".

So instead of ben_w saying

>nobody really knows what "consciousness" means

you just changed it to

>nobody really knows what "internal experiences" means

So then we haven't really gained anything in terms of being able to do rigorous science w.r.t. consciousness (except for some vague pointers, which might be valuable nonetheless).

Indeed.

An additional problem is that "a camera pointed at a TV displaying the output of that camera" meets that definition of consciousness.

I don't think that setup should be counted as conscious.

It doesn't.

It transforms code from its training dataset in a sufficiently vague and exciting way. That's not all of what writing is.

Any sufficiently advanced technology is indistinguishable from magic.

> It transforms code from its training dataset in a sufficiently vague and exciting way. That's not all of what writing is.

For the purposes of this level of the thread, I don't see how it matters what's going on under the hood, just that it can output code when prompted?

(I'm pretty directly saying "consciousness is ill defined", after all).

> I don't see how it matters what's going on under the hood

And that is what I call "religious thinking" or "mysticism".

That's the entire point I was making. ML isn't intelligence. It's data transformation. But calling it "artificial intelligence" has become something of a religion to itself. Just like with any other religion, belief demands ignorance.

That's an odd direction to take.

You know Searle's Chinese Room? I'd have agreed with you if you'd just written that ML is that room.

I'm also saying fairly explicitly that right now it's impossible (no possibility of Popperian falsifiability) to say if an artificial neural net is or isn't conscious, because we don't know what that word really means.

But, the well-defined common definitions of intelligence are: abstraction (trivial CS), logic (fundamental), learning (all ML is this), reasoning and critical thinking (appears to be demonstrable but might be a Clever Hands illusion), planning (university courses cover this before they get to ML), and problem solving (widely demonstrated in multiple fields).

Again, I'm specifically avoiding the ill-defined parts of the definition. If you want to have a go at defining them, by all means, have fun — but my experience has thus far been that every definition falls into a few categories:

(1) recursive meaninglessness (think "intelligence is consciousness, consciousness is intelligence")

(2) things which humans fail at (think of a test of intelligence which, if applied to a human, would be equivalent to claiming a human born without eyes wound really know what the subjective difference between red and green is)

(3) things which computers can do trivially (sometimes even finite state machines, let alone ML systems)

So yeah, not sure why you think "intelligence" is religious and why textual output isn't "writing".

Again, you are appealing to ignorance.

Confusion can't take the place of knowledge.

We don't know how much of a replacement for intelligence we have created with ML, but we can be certain it isn't a complete one. No matter how difficult it is to articulate, the difference between ML and intelligence is vast and obvious.

TBH, other than what name you want to give to things, I think you're agreeing with me without realising it. I'm not appealing to anything, rather I agree with every other specific you've written besides your first sentence.

A nice demonstration that consciousness does not necessarily result in self-awareness, bravo.

https://mitpress.mit.edu/9780262561273/the-computational-bea... and

https://github.com/gwf/CBofN

On Debian/Ubuntu: install xorg-dev build-essentil git and clone it.

*BSD users: You and we already know what to do.

Note to OpenBSD users: the examples will crash CWM on some machines, use FVWM for the examples, or run Xephyr on a slightly lower res (800x600 will perfectly work against a 1366x768 resolution, these are old programs), export $DISPLAY to :1 and then run the examples.

Whose viewpoint do you think you’re making fun of here?

  Searle's Chinese Room?

Say, you put three individuals in the room. Say, Steve Wozniak, Steve Jobs, Steve Balmer. Say, the experiment is plus or minus Jack Ma. The room follows the script. I guess 30 minute runs of the experiment generates a kind of collective consciousness in the conversation content between participants that is completely off script.

Reply on news.ycombinator.com