> Douglas Hofstadter had this whole theory of consciousness and when he found out that an LLM was a simple feed forward network with no feedback loops he went into a crisis. Basically his whole theory in GEB was wrong, according to him.

LLMs are not conscious. The training process for an LLM is not a feed forward network. If we were going to try to fit the idea of consciousness a la humanity (which is really the only fully 'conscious' creature we know of) into LLMs, then 'running' an LLM is identical to cloning a frozen human, thawing it, firing some neurons, reading the result and then destroying the clone.

A better argument for actual consciousness would come from the training process, but that itself is also dubious. It's unlikely consciousness is an emergent phenomenon. Or rather, such a claim is extraordinary and would require an extraordinary amount of proof, which GEB does not provide, sorry.