In a new book, Christof Koch views consciousness as a theorist and an aficionado
arstechnica.com
arstechnica.com
I think one would have to be unaware of the ability of simple Turing-equivalent machines to simulate devices of much greater complexity than themselves in order to find this argument persuasive. We see here that Koch is a fan of IIT; can he show one machine with a high IIT score that cannot be simulated by a simple (though sufficiently-sized) Turing-equivalent computer, possibly abetted with a source of entropy?
You can write a computer program that simulates pain. Perhaps even to the point that it convinces people that it feels pain. But, actually, it doesn't have a nervous system and can't feel pain.
Maybe in the future we'll build machines that can feel (and are conscious) but currently we don't know how. It would be very strange if we build those machines accidentally while training LLMs.
https://www.101computing.net/enigma-machine-emulator/
I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up.
What does matter? Does consciousness? It may matter less than is widely assumed since LLMs can do lots of cool stuff and aren't conscious.
> I agree that we don't know how to build machines that can feel (and are conscious); all I am saying here is that Koch's argument (at least as presented in this article) against a suitably-programmed transistor-based, von Neumann architecture digital computer becoming a conscious machine does not hold up.
I agree that his argument isn't very good. Yet I'm sure it's the case that there is no possible program that can run on any computer that exists today that can be conscious.
Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build. If we want to build it, we'll almost certainly need to understand what makes humans or other animals conscious. The odds of just stumbling across the solution seems very low.
Personally, I think it does, and I do not think it can be dismissed as either epiphenomenal or some sort of illusion. I don't think LLMs are evidence against this, as, while they are uncannily like humans in some respects, they seem to lack some pretty important things (such as self-recognition or a theory of mind, or, for that matter, any concept of language as referring to an external world that goes by rules independently of what is said about it.)
> Hardware matters. Computers need multiple cores to do parallel computations. Similarly, they'd need some kind of hardware to be conscious, hardware that we don't know how to build.
This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
> This is not a valid argument - the conclusion ('[computers would] need some kind of hardware to be conscious, hardware that we don't know how to build') does not ineluctably follow from the premise ('Computers need multiple cores to do parallel computations' or even just 'Hardware matters'.)
I'm not claiming that my argument proves anything. The point I'm making is that simulating something in software isn't magic. You can't write a computer program that executes code in parallel and run it on a single core CPU and observe parallelism. You need to have hardware that supports parallelism.
I think it's very likely that this is also the case for consciousness. You can simulate consciousness. But to actually achieve conscious experience, you need a certain kind of hardware. This is just my opinion, there's no proof here.
That's a controversial, though probably majority position, among people who study consciousness. Functional equivalence in the minds of many hard-nosed, very important folks in the field *does* entail feature equivalence with respect to consciousness.
Fundamentally, this is a dichotomy between objective and subjective. Functional is an objective measurement.
What makes you sure of that? I have read all the usual arguments, and I don't find them sufficient to justify the certainty that it is beyond the realm of possibility of doing so, at least up to the standard of evidence by which we assume other people are experiencing something.
Framing the physical-mental problem in terms of equivalence is essentially begging the question anyway, tacitly presupposing that the mental is not a physical phenomenon. No-one asks what, physically, is equivalent to a hurricane (to use just one example of a complex and what was once a mysterious phenomenon.)
Genetic algorithms, for example, given enough time, will optimize differently on different hardware, or even in a different environment. What's amazing is how resilient organic creatures are, humans in particular, to changes in "identity."
I was going to quote-dunk a bunch of it, but I think it's beyond the point -- no one's going to be convinced on the scientific validity of "consciousness" in this thread alone. I will say that I recommend Patricia Churchland's Neurophilosophy and Noam Chomsky's [Language and The Mind](https://www.ugr.es/~fmanjon/Language%20and%20Mind.pdf) for scientifically-minded views that don't share this persons goal.
Btw from the Chomsky book you linked:
> > The technological advances of the 1940s simply reinforced the general euphoria. Computers were on the horizon, and their imminent availability reinforced the belief that it would suffice to gain a theoretical understanding of only the simplest and most superficially obvious of phenomena – everything else would merely prove to be “more of the same,” an apparent complexity that would be disentangled by the electronic marvels
There is, as of yet, no scientific justification for thinking that humans are different from computers in some fundamental physical sense. We can talk about differences in architectonic structure like this article does (densely connected neurons v sparsely connected transistors), but ultimately each neuron is a machine, and at this point they’re machines we understand pretty darn well. Like all other machines bigger than 0.0001nm or whatever, there’s not really a clear mechanism by which they could be meaningfully “controlled” or “influenced” by quantum interactions.
In this light, the answer is simple: human brains are incredibly complex machines. Some of the main counter arguments are:
1. It “feels” like “something” to be “you”, which is a whole field science lacks the terminology/framework to study. This is the position taken by this author, more or less
2. All conscious beings so far (animals included) have been biological, so simple parsimony could argue that we need a counterexample first.
3. Humans are able to communicate with each other telepathically by tapping into an unknown physical field. Obviously won’t win a lot of fans on this forum, but it’s an interesting piece of trivia that this final point was the only one Alan Turing found plausible in his famous 1950 paper Computing Machinery and Intelligence. See page 17: https://courses.cs.umbc.edu/471/papers/turing.pdf
This is similar to the question of, can we simulate a brain? Even if we can't build one using a LLM-type construct, it seems at the very least we could model atomic interactions in detail and brute-force a simulation that way. However I think this requires that there is a lower limit to the level of detail of physical reality; otherwise the non-determinism such as what you see with the uncertainty principle becomes intractable. As a result it's not clear to me at all that humans are not different from computers in a physical sense.
It seems insane to me to believe that consciousness can emerge from this mess, simply because "you performed the right computations", regardless of cpu architecture or operating system or programming language or whatever. If consciousness really could emerge from all this, interpret the CPU instruction set, parse virtual memory tables, dereference pointers, decrypt and uncompress memory, parse Unicode text, assemble all of the necessary information scattered all about in the system, then consciousness is magic, a miracle.
Or, we could simply assume there's some unknown physical mechanism/process/activity in the brain that leads to conscious experience. Almost everything else in the universe is the result of a physical process, magnetic fields, a nuclear bomb exploding, even quantum entanglement. Why should consciousness be an exception?
Those atoms and molecules may just be bits of information themselves in some higher order computer. But there is nothing inherently "magical" about consciousness other than its uniqueness.
Heck there are fairly convincing arguments for pan-consciousness where it is a fundamental part of any set of information and is simply as complex as that information system. If you have a highly complex, self referential information system like our brains, then the complexity of consciousness is equivalent to our experience. The chinese box would have its LLM-like consciousness, which we would not recognize as our own, but could still a qualitative experience born from objective information states.
Think more about your last paragraph, it undermines your argument from the previous two. If everything is the result of a physical process, then how is that an argument for consciousness being somehow fundamentally different or exceptional in our ability to recreate or simulate it?
Yes, I agree with that much. Hopefully I understand you, but I do believe we could create a "consciousness chip", so to speak, that performs the proper physical process to create real conscious experience, not just simulated. But it's unlikely to happen with existing computer hardware. (by "create", I'm not saying that consciousness emerges from nothing. Rather, there's this idea of "activating" consciousness, "turning the lights on".)
> Those atoms and molecules may just be bits of information themselves in some higher order computer. But there is nothing inherently "magical" about consciousness other than its uniqueness.
I don't buy the idea that the universe is a computer nor that we're living in a simulation. I do believe there is an objective reality.
> Heck there are fairly convincing arguments for pan-consciousness where it is a fundamental part of any set of information and is simply as complex as that information system. If you have a highly complex, self referential information system like our brains, then the complexity of consciousness is equivalent to our experience. The chinese box would have its LLM-like consciousness, which we would not recognize as our own, but could still a qualitative experience born from objective information states.
It's an idea that comes out of sci-fi. It's been used as a plot device in some episodes of Star Trek. But it's just that, science fiction.
> Think more about your last paragraph, it undermines your argument from the previous two. If everything is the result of a physical process, then how is that an argument for consciousness being somehow fundamentally different or exceptional in our ability to recreate or simulate it?
That's not what I said. What I'm saying is computers are not magically conscious. Could we recreate consciousness with the right hardware? Sure, I don't see why not, like what I said with a "consciousness chip" . But is consciousness magically going to emerge purely from a simulation without any special hardware? No, of course not.
I'm not talking about hand-wavy sci-fi or spiritual "the universe is a sentient being" stuff. I'm talking about one of the few solutions to what the nature of qualia is and why we aren't philosophical zombies. Why would the universe have some mechanism of consciousness available to it prior to those structures even existing? What is more convincing about a physical process being the literal mechanism for consciousness and not an informational process that has an abstracted physical basis? Could the literal mechanism not just be the information system considering you can create the same behaviors in a simulation as in reality?
Consciousness being some thing that just appears in the universe all of a sudden because evolution happened on this "neat trick" in physics seems more "magical" than it being an emergent property of increasing more complex systems, but a property that has always been there. We see the same emergent behavior appear in all sorts of systems built using different parts and/or simulated as long as they are in the same context of rules. Why wouldn't that also hold true for the emergent behavior of consciousness?
The argument for it being this unique thing that only can happen via a strict physical process strikes me as dualist. Why is it a unique thing?
There's a physical basis for most things. A magnetic field forms because you have a bunch of particles with the same quantum spin. A stove burner glows red when you run electricity through it.
I'm not saying the process in the brain is unique. I'm not saying there's only one way to "activate" consciousness. Heck, for all I know, maybe a bolt of lightning experiences consciousness, even if just for a fraction of a second.
Allow me to give you a "physical" example. The brain operates near a critical threshold, that is the edge of chaos. The brain is teetering on the edge of chaos without ever going chaotic. Well, not usually, because when the brain does enter chaos, you get a seizure. But there is growing evidence that this may be a prerequisite for consciousness; when the brain enters a less chaotic state, a person may lose consciousness.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8851554/
Now let's talk about information processing. Most of what our brains do is subconscious. Only a small subset of what our brains do is actually conscious. What makes some information processing more special than others that some of it is conscious and the rest is not? Isn't it more likely that the way the information is processed is what makes the difference? I.e. different at a physical level.
> Only a small subset of what our brains do is actually conscious. What makes some information processing more special than others that some of it is conscious and the rest is not?
This is an excellent way of approaching the question, but I just as easily can say isn't it more likely that the difference is the pattern of the information and not the strict physical structure that makes it? Look at how many different physical structures and mechanisms we have for seeing, hearing, breathing, touching, etc across nature. Many of them are fundamentally different from each other, but end up in the same result of a sense.
Isn't it more likely that conscious thinking is like other senses in that it's a kind of information processing, rather than a specific mechanism of processing?
This also make it more likely to answer your question of why are some mental processes conscious and the majority are not, it would seem far more likely that the brain's neuronal structures (most of which are the same basic cell throughout the brain, just in different types of structures) discover different patterns rather than fundamentally different physical processes.
You can construct a computer using purely mechanical parts - gears, cams, springs, etc. The first calculators were purely mechanical.
So imagine you built an absolutely massive computer, made purely out of mechanical parts, that was capable of performing matrix multiplication and therefore running LLMs. It's technically computation, but should we seriously expect real consciousness to emerge from a purely mechanical computer just because it performed the right computations? Of course not, that would literally be a miracle.
And yet a bag of proteins, mitochondria, water etc. somehow isn't?
> The point is, consciousness doesn't emerge purely from computation alone.
[citation needed]
> It's technically computation, but should we seriously expect real consciousness to emerge from a purely mechanical computer just because it performed the right computations?
Of course yes. I don't see why consciousness couldn't theoretically emerge even from computations made by a really bored human placing stones on an endless beach, or in an anthill with any single ant being completely oblivious of what's going on. Such consciousness will only be conscious of data that's provided to it as its input, but conscious nevertheless - just like we are only conscious of a specific representation of the real world as perceived by our flawed senses. I don't expect any meaningful difference there, other than our minds being at least a few steps ahead of anything we can build at this moment, which is hardly surprising given how little we know about brains and bodies even today.
Computation isn't magic, even when we barely understand it.
Lord give me strength.
> Koch suggests that exercise, meditation, and the occasional guided psychedelic might be beneficial to many people. Substances such as psilocybin can enhance one’s feeling of well-being and facilitate pure presence. It is rare to have a close brush with death, and mystical transformations typically come unbidden; in contrast, psychoactive substances—though not entirely predictable—are more consistently available and can be managed safely. Koch’s book implies that there’s immense psychological benefit of entering "the flow" through chemistry that might outweigh the small risks involved.
This is cool. I like that he considers states of consciousness for the everyman and is not talking about philosophical zombies (Angels on a Pinhead for the modern age).
Lots of people putting weight behind it, but also with a degree of hubris similar to the string theory guys of the late 90's. But just like string theory, it also has a lot of fair criticism and is mostly still in the hypothesis phase.
A "clearer" study would require starting from some kind of zero. But nobody wants to go live in a cave and whoever did might not be inclined to write an article about it.
What I think will work, is to examine all the different forms of consciousness — including not just animals, but also those humans we find unfathomable, alien, and impossible to empathise with.
On consciousness, I'm interested on the integrated information theory.
Has he been living under a rock? Modern AI models already outpace the connectivity of the human brain, and are only getting bigger.
Modern estimates are that there are 100T neuronal connections in the adult human brain [0]. And that’s neuronal connections alone.
Astrocytes also make direct connections with neurons and can modify and induce neuronal activity [1]. There are 100B neurons [0] and ~20B astrocytes in the adult human brain [2, 3].
So this 100T connections estimate is only a small slice of the picture of human brain activity.
[0] https://medicine.yale.edu/lab/colon_ramos/overview/#:~:text=....
[1] https://neuraldevelopment.biomedcentral.com/articles/10.1186....
[2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5063692/
[3] https://link.springer.com/article/10.1007/s00429-017-1383-5
"In human brains there are about 100 billion neurons (nerve cells), each capable of producing an electrical pulse up to perhaps a thousand times a second."
So maybe that's wrong. Thanks for the links. Even so if we look out 10, 100, 200 years from now, that level of complexity will be greatly surpassed by AI.
It is astonishing that we can mimic human reasoning so well with these LLMs, but the essence of cognition seems to still be missing.
I agree with your belief that human processing will be surpassed in the future. We do live in some exciting times :)
Granted, you would be doing it with a frame rate limited by the processing power of the computer, but that just means that a thought that takes a human 1 second to arrive at might take much longer for the AI (for now).
But at this point the TYPE of computation performed by neurons, which is unlike what modern computers do-- for example, brains do not appear to have addressable memory units separate from compute units-- do seem to be differing enough to perhaps explain some of the gaps between computers and minds. Some even think neural computation is a different category of computation altogether: https://pubmed.ncbi.nlm.nih.gov/23126542/
And the key point is this- these models are the worst they will ever be, and are gaining size at pace. So even if you we grant the argument that our brains are still a bit more complex, hopefully we can agree that will not be the case in 5 years. Heck, how about 20 years, or 100? Let's be real.
In any event, raw parameter/weight count to me seems like a very primitive way to judge "complexity" in comparison to the human brain. Looked at most ways, our brains are for more efficient at doing the incredible things they do than LLMs. Consider how little language young children are exposed to in comparison to LLMs given their abilities to figure out how to produce language.
If the brain doesn't work like an LLM, you can expand the size and "complexity" of these models to the moon and they won't outperform the brain. Current models can write impressively well, but they can barely do math. It's clear they don't reason as we do.
google recently scanned a 1mm cube of human brain which was 1.5Petabytes of raw data. The AI hardware that Google trains on is multiple racks.
I think a better analogy would be between an entire google datacenter (including all the networking, storage, sensors, processors, and memory) and a human body although even then it's a stretch.