Comment by dokein | Hacker News Reader

Comment by dokein | Hacker News Reader

With respect to Iain Banks, he's probably one of my favorite authors (both sci-fi and non-sci-fi).

With respect to the question of having consciousness and free will -- don't we all just reduce down to atoms obeying the laws of physics? Are our actions not therefore deterministic? Sure maybe there's some quantum uncertainty principles at work (but it's not like LLMs produce the same output every time either). So it's not at all clear how consciousness arises, and thus not clear that e.g. GPT-7 "cannot" be conscious.

The other part of this is as relates to the question of AGI and its risks -- there's lots of things that are not necessarily consciousness but are nonetheless extremely bad for humans. Viruses. Tigers. Fire (when not appropriately managed). And for the set of things on Earth we're pretty well equipped to handle them, but if the sun went supernova, screaming "nooo I'm conscious and you're not" would not prevent humanity from being wiped out.

> So it's not at all clear how consciousness arises, and thus not clear that e.g. GPT-7 "cannot" be conscious.

Arguments from position that there is an essential human characteristic that AI cannot capture always leaves me wanting. It seems almost like their arguing from a position of ontological impossibility when I read their responses.

On one had we have a few simple classical physics equations which describe how matter behaves, and brains being matter, should follow these rules. On the other, we have universal function approximators - models which given enough resources can approximate any function. Theoretically it seems like one can be arbitrarily approximated by the other.

That's why dismissals feel like an argument from consequence. As one gap closes, another gap emerges. In the same way that science creates a god-of-the-gaps - incremental advances in science make god retreat into ever smaller and obscure gaps, advancements in AI create a consciousness-of-the-gaps where the justifications for consciousness retreat into ever smaller and obscure gaps. Embodiment is the newest of these gaps and once embodiment occurs, another gap will be discovered and retreated to.

Unfortunately, consciousness and sentience are social categories, not scientific categories - like countries, they gain legitimacy when enough agents say they're legitimate - and just like the gaps, this trend isn't stopping.

> With respect to the question of having consciousness and free will -- don't we all just reduce down to atoms obeying the laws of physics? Are our actions not therefore deterministic?

The majority of living humans profess a religious faith and probably would not agree with that statement. That doesn't make it wrong obviously, but I don't think you should treat it as a settled question.

Reply on news.ycombinator.com