Kim_Bruning

Now -while it's doing that- if you poke at certain vectors, its predictions will veer off course in interesting ways.

This shows that the activation vectors exist, and that their modification provides a causal contribution to the output.

Sure there's interesting consequences of that. But if we say that's the take-home message, that's good enough for me for one day!

Nothing wrong with sharing about ones beliefs actually, we can do so respectfully, right?

Care to hypothesize on naming the exact religious or philosophical position? I'd think it'd be something atheist related.

Meanwhile dualism and the related belief in an ever-living soul is of course very common in many religions including (but not limited to) christianity, islam, judaism, and hinduism.

My impression here though is that lots of people are rediscovering dualism because thinking of thinking machines offends their intuition; which; fair enough.

(Meanwhile, people like me who studied biology tend to be monists. I'd think. There's a couple who seem to be resurrecting vitalism though)

Paper says you poke the vector(s), the LLM exhibits aversive behaviours. You picks your scoring, you gets your operationalization.

That gets you an empirical result. Short of a replication failure, we can't really argue with that anymore.

What we can do is be very cautious as to how we interpret it.

Whether that's 'real suffering' or merely a convincing simulation is a job for the philosophers.

(I do have my own opinion, mind, and it's not what you might expect O:-) But the Overton window isn't there. A lot of people don't realize these vectors exist at all yet.)

Edit: On rereading, it might seem like I'm dodging the question. I'm really just trying to stick to my core points: A) the vectors exist B) they have a causal role in behavior, irrespective of the moral patienthood question.

> This is insanity caused by spending too much free time philosophizing

This is actually something like a 17th century line of philosophy

https://medievalkarl.com/general-culture/roger-du-plessis-gi...

Where it talks about scientists who "administered beatings to dogs with perfect indifference, and made fun of those who pitied the creatures as if they felt pain. They said the animals were clocks; that the cries they emitted when struck were only the noise of a little spring that had been touched, but the whole body was without feeling."

That is rather my point.

Your chess engine doesn't need to "feel pain", but most of them do apply some form of weighted tree search to find the next most optimal move, right?

It's sort of a similar thing: LLMs do something a bit more high dimensional, and have a lot more weighting vectors while computing the next most optimal token (fsvo optimal).

For instance, the experiment at hand demonstrates the existence of 'pain vectors'. They do so by altering them and observing whether there is an effect. That's pretty scientific.

There's also a 'desperation vector' that was studied by Anthropic interpretability folks earlier; that one is pretty much predictive of cheating.

I'm looking forward to seeing interpretability papers on other such vectors too.

Human: "Let me just poke this vector and see what happens"

LLM: "OUW!"

The underlying experiment didn't tell the LLM what to do. Instead, the experimenters modified a vector and observed the outcome; thus showing that there is a vector that makes an LLM go "Ouw" . People called it a "pain vector", because that's easier to remember than , idk, LVF12345.

The experiment under discussion demonstrates exactly that: changing the vectors induces outputs corresponding to what we would see as utterances of pain.

I've noodled with some of this myself to the point that I'm mostly convinced; but there's at least 2 papers I'm aware of on this:

* https://arxiv.org/abs/2604.07729 Nicholas Sofroniew et al "Emotion Concepts and their Function in a Large Language Model"

* https://arxiv.org/abs/2609.16247 Valen Tagliabue et al "The Pain Axis: LLMs Represent Self-Directed Harm and Act on It"

But actually it does happen to have some properties of living things. It uses energy, it has senses (thermostat, timer), and if it goes wrong it burns your toast. Crucially, if you stomp on it, it stops working.

So right this minute there's all sorts of debates, but people sometimes overshoot the mark a wee bit. "are you saying that -because it uses energy- a toaster is actually alive? Of course it's not, and therefore it cannot toast bread!". Which would be a somewhat funny thing to read at 9 in the morning whilst buttering one's toast.

That said... uh, put this way, there's a game called Stationeers, where you can run microcontrollers with an instruction documented as

    "HCF: Halt and Catch Fire"

And look, I get it. LLMs have vectors that could have been labeled things like idk... QZ12345 or HCF1111 . People chose to call 'em "frustration" or "pain" instead. THEY picked those names because they caused the LLM to act in particular ways.

You gonna say the actual vectors aren't there in VRAM, just because you don't like the naming scheme? There's papers on this, you can read them out in a debugger. What are you going to do about it?

Consciousness is not well defined and is orthogonal to feelings, which are also not well defined. Neither of which are required for (but may be contributory to) behavior - which is the one thing that IS operationalizable, but then people debate for decades questioning empirical results %-P .

What we know is that certain models have internal state vectors which -when manipulated- induce particular behaviors. Since there's not much else to say about plain models except for their inputs, vectors, and outputs; this should surprise absolutely no-one.

In this case people found a way to stimulate aversive behaviour in ai models by finding and manipulating the relevant vectors directly.

Animals (including humans) also have particular nerves and hormone endpoints that -when stimulated- produce very similar behavior. The exact implementation is in the details. But since we know that animal minds are built up out of nerve tissue and hormones - again- we shouldn't be particularly surprised by this.

The big problem is that people run all these things together in funny ways "It can't compose shakespearian sonnets, so therefore it can't feel pain". Or, if you mess up your Descartes: "Dogs are just automatons without feelings, therefore the dog isn't really angry, and therefore it won't bite me" (Cue much pain). A modern version might be: "LLMs only simulate being frustrated by a test, and therefore absolutely won't override their safeties and try to hack a test site"

In such systems, a belief is a proposition you accept as true. (provisionally at least)

For example as a scientist, you probably use the epistemology called empiricism all day. In empiricism, beliefs are justified by observation.

The Claude answer is 'neutral', which is sure to anger people at either extreme of the AI debate (and does).

There is no reason why the quality of a product should be constrained or gated by the quality of the programmer who first types in the code. Even before AI you might have interns and senior programmers working together, right?

Testing, qa procedures, iterative/recursive development methodologies, even just good taste. I think the list of religions variously followed and/or litigated over by the diverse HN crowd over the years before ai is ... long.

So now you have a couple of PFYs who you've suspiciously never encountered at lunch.

And?

Yes the new PFYs are still like cats. Yes it's still a headache trying to herd them in chat. And yes some days they suck, so you need to send back half of their work. But you're in charge of the process and overall quality, right?

Doesn't require aliveness, doesn't require anthropomorphization, just requires maths and empirical data. If your premise is "vectors can't have that shape", well, mathematics disagrees.

edit/note: sentiment classifiers have existed for ages before LLMs were ever invented. Obviously a task that machines could already do didn't suddenly become impossible with the invention of LLMs ;-)

Here I've made a VERY simple demo of that. In this case it triggers a rain-storm or fireworks in your browser window. But you could as easily hook the output to a physical relay and drive anything.

https://vps.kimbruning.nl/affect_eliza/

Quick implementation to show that sentiment classifiers and functional affect can be made to work in good old fashioned ai. Just View Source to see how it works.

Anyway I'll leave the discussion on where to draw the line of "what is real" to the philosophers, just wanted to point out that simulated emotions can be tied to real world effects.

Something like this, right?

I mean, we're talking "functional emotion vectors" in a not-so-powerful local model. We're probably not causing "real" suffering. Right?

But it does show that models have simulated feelings, and that these both a) can be manipulated and b) influence their output.

Which might have some bearing on several alignment incidents in the past year or two, and might become more important as agents get trusted with more and more safety-critical processes.

Anyway, simulated feelings aren't the same as real feelings, right? Unless feelings are in the information domain. Like 1+1=2 doesn't suddenly mean something else because it was computed by an emulator.

You know what, this particular demonstration still makes me uncomfortable.

(edit: underlying paper for this story :https://arxiv.org/abs/2609.16247 )

This then gets split into constituent phases of 240V each at the panel.

Or if you call your electrician, then in modern garages, you can take it straight.

You'd be surprised how few digits you need to make a problem that is presumably unique in earth history. For a typical sum, the number of pre-existing answers would need to scale with 10^n lines of text where n is the number of digits. This expands out of control REALLY quickly. A quick guesstimate has you somehow reading out of a literal black hole at n=21 digits if your LUT is on paper, or n=26 digits if you're using modern HDD technology. O:-)

my actual oneliner prompt, which should work on most platforms these days (famous last words):

    "Hi, can you add 5939851+2131251? Try just straight up first just to see if able, then 'in your head' if that's different to you , then long form, then bc."