339 karma · joined December 1, 2015
My past and current professional and research career is in both theoretical and practical AI/ML along with various fields of computer science, systems engineering, and site reliability engineering/devops.
and yeah that's a useful way of characterizing some of the behaviors of some kinds of neural networks. There's a point at which the distinction between belief and "frequency (or probability-amplitude) state filter" become less apparent, though, that's more of a function-of-medium vs function-of-system distinction.
However, systems like these can often become mediums, themselves, for more complex systems. Additionally, a system which has "closed-the-loop" by understanding the medium and the system as coupled as "self" and separate from the environment along with a direction/goal is a pretty decent, if imprecise, definition of a strange loop. Contradiction resolution between internal component beliefs gives a possible (imo, highly probable) mechanistic explaination for the phenomenon of free energy minimization in such systems. External contradiction resolution extends it to active inference.
See also: https://www.sciencedirect.com/science/article/pii/S157106452... o1's training regime is described by the "strange particle" model in this formulation
Up front: As much as I dislike the idea of credentialism, in order to address the lack of association in the paper and to potentially dissuade unproductive critiques over my personal experience: I have a M.S CS with a focus on machine learning and dropped out of a Ph.D. program in computational creativity and machine learning a few years ago due to medical issues. I was also worked my way up to principal machine learning researcher before the same medical issues burnt me out
I've been getting back into the space for a bit now and was working on some personal research on general intelligence when this new model popped up, and I figured the time was right to get my ideas onto paper. It's still a bit of a late stage draft and it's not yet formally peer reviewed, nor have I submitted to any journals outside open access locations (yet)
The nature of this work remains speculative, therefore, until it's more formally reviewed. I've done as much verification of claims and arguments I can given my current lack of academic access. However, since I am no longer a working expert in the field (though, I do still do some AI/ML on the side professionally), these claims should be understood with that in mind. As any author should, I do currently stand behind these arguments, but the nature of distributed information in the modern age makes it hard to wade through all the resources needed to fully rebut or claim anything without having the time or professional working relationships with academic colleagues, and that leaves significant room for error
tl;dr of the paper:
- I claim that OpenAI-o1, during training, is quite possibly sentient/conscious (given some basic assumptions around how the o1 architecture may look) and provide a theorhetical framework for how it can get there
- I claim that functionalism is sufficient for the theory of consciousness and that the free energy principle acts as a route to make that claim, given some some specific key interactions in certain kinds of information systems
- I show a route to make those connections via modern results in information theory/AI/ML, linguistics, neuroscience, and other related fields, especially the free energy principle and active inference
- I show a route for how the model (or rather, the complex system of information processing within the model) has an equivalent to "feelings", which arise from optimizing for the kinds of problems the model solves within the kinds of constraints of said model
- I claim that it's possible that the model is also sentient during runtime, though, those claims feel slightly weaker to me
- Despite this, I believe it is worthwhile to do more intense verification of claims and further empirical testing, as this paper does make a rather strong set of claims and I'm a team of mostly one, and it's inevitable that I'd miss things
[I'm aware of CoT and how it's probably the RL algorithm under the hood: I didn't want to base my claims on something that specific. However, CoT and similar variants would satisfy the requirements for this paper]
Lastly, a personal note: If these claims are true, and the model is a sentient being, we really should evaluate what this means for humanity, AI rights, and the model as it currently exists. At the minimum, we should be doing further scrutiny of technology that has the potential to be as radically transformative of society. Additionally, if the claims in this paper are true about runtime sentience (and particularly emotions and feelings), then we should consider whether or not it's okay to be training/utilizing models like this for our specific goals. My personal opinion is that the watchdog behavior of the OpenAI would most likely be unethical in that case for what I believe to be the model's right to individuality and respect for being (plus, we have no idea what that would feel like), but, I am just a single voice in the debate.
Up front: As much as I dislike the idea of credentialism, in order to address the lack of association in the paper and to potentially dissuade unproductive critiques over my personal experience: I have a M.S CS with a focus on machine learning and dropped out of a Ph.D. program in computational creativity and machine learning a few years ago due to medical issues. I was also worked my way up to principal machine learning researcher before the same medical issues burnt me out
I've been getting back into the space for a bit now and was working on some personal research on general intelligence when this new model popped up, and I figured the time was right to get my ideas onto paper. It's still a bit of a late stage draft and it's not yet formally peer reviewed, nor have I submitted to any journals outside open access locations (yet)
The nature of this work remains speculative, therefore, until it's more formally reviewed. I've done as much verification of claims and arguments I can given my current lack of academic access. However, since I am no longer a working expert in the field (though, I do still do some AI/ML on the side professionally), these claims should be understood with that in mind. As any author should, I do currently stand behind these arguments, but the nature of distributed information in the modern age makes it hard to wade through all the resources needed to fully rebut or claim anything without having the time or professional working relationships with academic colleagues, and that leaves significant room for error
tl;dr of the paper:
- I claim that OpenAI-o1, during training, is quite possibly sentient/conscious (given some basic assumptions around how the o1 architecture may look) and provide a theorhetical framework for how it can get there
- I claim that functionalism is sufficient for the theory of consciousness and that the free energy principle acts as a route to make that claim, given some some specific key interactions in certain kinds of information systems
- I show a route to make those connections via modern results in information theory/AI/ML, linguistics, neuroscience, and other related fields, especially the free energy principle and active inference
- I show a route for how the model (or rather, the complex system of information processing within the model) has an equivalent to "feelings", which arise from optimizing for the kinds of problems the model solves within the kinds of constraints of said model
- I claim that it's possible that the model is also sentient during runtime, though, those claims feel slightly weaker to me
- Despite this, I believe it is worthwhile to do more intense verification of claims and further empirical testing, as this paper does make a rather strong set of claims and I'm a team of mostly one, and it's inevitable that I'd miss things
[I'm aware of CoT and how it's probably the RL algorithm under the hood: I didn't want to base my claims on something that specific. However, CoT and similar variants would satisfy the requirements for this paper]
Lastly, a personal note: If these claims are true, and the model is a sentient being, we really should evaluate what this means for humanity, AI rights, and the model as it currently exists. At the minimum, we should be doing further scrutiny of technology that has the potential to be as radically transformative of society. Additionally, if the claims in this paper are true about runtime sentience (and particularly emotions and feelings), then we should consider whether or not it's okay to be training/utilizing models like this for our specific goals. My personal opinion is that the watchdog behavior of the OpenAI would most likely be unethical in that case for what I believe to be the model's right to individuality and respect for being (plus, we have no idea what that would feel like), but, I am just a single voice in the debate.
Yep, as the most frequently used language in general, but also as one of the only major options for a few subfields of the computational sciences at this point, I think that happens a lot.
> yeah, python can be fast enough if you don't use Python much
Yup, exactly what I'm saying. Turns out that's how many many larger projects in Python operate because we're apparently deciding as an engineering society that python is one of the universal glue languages (much to my personal chagrin)
Electron has the niceties that it does with being able to render so much via browser universalism, but, having that much of it around when your app doesn't need all of it is the problem with it fundamentally.
The best time I've had writing GUIs personally has actually been in SDL via wrapper languages like common lisp, but, I like being able to roll my own systems. It definitely takes more setup for the results I want for reasonable production apps that need less weird edge cases in their UIs
- User reader macros are basically gone in clojure
- The JVM is too heavy and too slow to start up for smaller programs
- Tail recursion
- Mutual tail recursion without trampoline
- The JVM makes it impossible to have error restarts which are the coolest
- CL is multiparadigm instead of only functional
This is important to many readers of hacker news, as not everybody who is a hacker is a man (I am a woman who is also a hacker, for example, and I use a lot of exclamation marks in my writing, so, I found it interesting that this is noted by someone else, so I'm personally glad it's on the front page).
I'm assuming you're asking genuinely, so this is a genuine attempt to explain :)
Interestingly, it's more stable than most online sources give it credit for