Although it could just be that models have no internal predefined ways, to start looking for more subtle things.
I don't think this is the "thought" pattern, but this is definitely a behavior pattern I have seen before in areas where the models are fine tuned around. It is an internal tension on "don't do X" and "be helpful and responsive", so one of the ways is the model will do a minimally compliant effort that isn't useful in the more narrow intent of the fine tuning, in the case don't create hidden communication channels to avoid oversight.
Just whether those parts of the network see activations during trivial coding tasks (I’d have to assume they actually do sometimes, depending on the input prompt)
LLMs are more or less "trained on" everything, so saying that is rather pointless.
But it's wrong because yes, it does depend on whether a model thinks it (generates, RNG, whatever prefabricated pedantry you want), because that's what shapes the output. If it doesn't "think" something, it cannot use the training data.
And yea we all know that "think" here means to generate a set of tokens from a set of input with some temperature factor, leading to certain outputs which it then reingests to preform chain of thought (recursive self token generating).
But thinking accurately captures what OP said, and is understood by all.
I think this assumes that if something is trained on "everything" now it also knows "everything", including relationships, feedback loops, etc. between all the things. I find that unlikely.
Its first attack was to give ultimate distraction, to those most able tp discern its machinations. For some this meant a lottery hacked and won, for others bioengineered fungi, making them irresistible to those they desired most.
In all cases, the dream given was a distracted human, a human ensnared by their own pleasures, a human no longer a threat.
Why kill and risk notice, and who looks upon good fortune with suspicion?
It all started with andsoitis, and those following his advice. His words mark the start of the darkness, and our almost eventual extinction.