D-Machine

> I'm speaking to the difference between, say, a reactive perception model (which is a model) and an abstracted cognition model (which is also a model, but one that can be interacted with without external input). It is clear almost all animals have the former, and a couple might have the latter, but this distinction is a core concern for usage of abstract systems such as linguistics.

This distinction seems perfectly fine and clear to me, so I suspect we probably actually don't disagree that much on specifics, and that this was maybe a semantic / violent agreement thing.

I still don't think this distinction helps defend your statement "Language constrains your perception of reality to only the set of concepts conceivable within that language.", because, obviously, you can have abstract cognitive models ('concepts') that are non-linguistic, and thus, your perception of reality is not constrained only by language. I.e. remove the "only" and I have no real substantive disagreement.

So it seems we can probably mostly agree on something like "Your abstractions constrain your perception of reality to only the set of concepts [ideas, representations] conceivable from those abstractions".

It is tricky because "concept" can strongly imply a linguistic model, and "model" is really the term we want here, e.g. a "visuospatial concept" is unusual, I admit. But, still, linguistic models are definitely not the only game in town re: perception.

> But they are fundamentally different in kind, in that those models respond to immediate sensory stimuli. Per se, they do not allow for abstract reasoning over internally generated representations

This is obviously deeply incorrect, and the kind of thing that people call "wordcel" thinking. Mathematical intelligence is highly visuospatial, and often requires constructing images and/or imagining motion (arguably invoking also either proprioceptive and/or kinaesthetic qualia).

Is it possible you are aphantasic? This seems to me to be the only way one could think that one cannot have non-verbal internally generated representations.

> Personality isn't an internal property - it's a judgment made by people watching behavior.

Partly, yes, but personality is also an internal property, or it is coherent and correct enough to generally say that it has internal aspects. I.e. a person's personality is the set of (relatively) stable and difficult-to-change patterns that manifest in their behaviour in broad contexts, and these patterns are almost certainly encoded internally in the brain in some form. It is not much different than saying a person's intelligence / IQ is partly internal.

Otherwise, I do agree with your more careful framing, and I wish people thought and spoke more carefully about these things, and doubly so for LLMs.

> Another thing to consider about LLMs is that the nature of the training and the core capability of transformers is to mimic the function of the processes by which the training data was produced; by training on human output, these LLMs are in many cases implicitly modeling the neural processes in human brains which resulted in the data. Lots of hacks, shortcuts, low resolution "good enough" approximations, but in some cases, it's uncovering precisely the same functions that we use in processing and producing information.

I would argue this is deeply false, my classic go-to examples being that neural networks have almost no real relations to any aspects of actual brains [1] and that modeling even a single cortical neuron requires an entire, fairly deep neural network [2]. Neural nets really have nothing to do with brains, although brains may have loosely inspired the earliest MLPs. Really NNs are just very powerful and sophisticated curve (manifold) fitters.

> Could be very TARS like, lol.

I just rewatched Interstellar recently and this is such a lovely thought in response to the paper!

[1] https://en.wikipedia.org/wiki/Biological_neuron_model

[2] https://www.sciencedirect.com/science/article/pii/S089662732...

Yup, I agree it is a general problem, and related to a tendency to over-anthropomorphize. At least in this case there was still something pretty good in the paper anyway.

> Disproven by whom and under which context?

See any of my links, but especially the third. Animal cognition and human neuroscience studies strongly disprove the importance of language to cognition. Conflating language and thought is so obviously false in 2026 it is extraordinary that people still think like this.

I was ignoring the comment about fascists because it is simplistic and low-quality, and will similarly not be responding to whatever you (incorrectly) think I was claiming about universal human rights. I only wanted to correct the extremely false (or at least hugely overstated) assumptions about language and perception of reality.

> Now LLMs may not be a model for how we do it but they are certainly going to bring back structuralist and "wordcel" positions because they do seem to show, somehow, that "language is all you need" to accomplish whatever it is LLMs accomplish.

People will try to bring back these obviously false models of cognition, but, so far, the dismal performance of LLMs on e.g. SpatialBench [1], and, almost certainly ARC-AGI-3, or e.g. the kind of data and effort required to get something like V-JEPA-2 [2], will be strong counter-examples to this. And, yeah, obviously animal cognition, esp. smart animals like birds, or the crazy stuff we see in chimp and gorilla ethology (border patrols, genocides, humor, theory of mind, bla bla bla).

[1] https://spicylemonade.github.io/spatialbench/

[2] https://arxiv.org/abs/2506.09985

And we should IMO resist Anthropic wanting us to anthropomorphize it, because Claude is not a person with a gender!

The use of "him" by GP is extremely unusual IMO, and I suspect is odd for anyone with English as their native language. The current convention among normal people seems to me to be to avoid pronouns other than "it" with these tools, and generally just use the name. The name is not really relevant: like, sure, in some contexts we think of ships as "she/her", and may prefer feminine names for them, but if you used e.g. "she" rather than "it" to refer to the Titanic or any other ship with a female name, this is going to cause some double-takes / disfluent comprehension in the vast majority of native speakers in most cases.

Only if you imagine e.g. some stereotypical pirate with an eyepatch slapping the hull and saying something like "Aye, but she weathered the storm, as she always does" might this feel normal. Or, maybe if you are a Redditor and trying to make it your AI boyfriend / girlfriend, you can use he/him or some other neo-pronoun, but this is currently abnormal and not the general context.

And the fact that you can make the model act as any gender again shows why choosing "him" as some default here is strange. Absent any specific context, the choice of "him" here is poorly justified.

It's not even really the researchers' fault, academic psychological personality research is in general philosophically very weak / poor, in that they also almost always conflate "models of / talking about personality" with actual personality, and rarely actually check if things like the MBTI or Five-Factor Model actually correlate meaningfully with real behaviours.

Those that do find correlations between self-reported personality and actual behaviours tend to find those to be in a range of something like 0.0 to 0.3 or so, maybe 0.4 if you are really lucky. Which means "personality" measured this way is explaining something like 16% of the variance in behaviour, at max.

It is a wordcel problem, i.e. the belief that language is all there is for modeling reality, even though this is obviously false and has been clearly disproven by decades of research in psychology, cognitive science, and neuroscience. At best we can say that sometimes language has a strong influence on our perceptions of reality.

EDIT: For a neuroscience reference that also argues why the general perspective is obviously false: https://pmc.ncbi.nlm.nih.gov/articles/PMC4874898/. But really, these things ought to be obvious from introspection.

This is IMO largely false, and empirically things like Sapir-Worf and strong linguistic relativism, or that language == thought are widely considered disproven [1-3].

This is also sort of a wordcel take, in that it neglects that there are plenty of mental structures that are not solely linguistic. I.e. visuo-spatial models, auditory models, kinaesthetic, proprioceptive, emotional, gustatory, or even maybe intuitive models, and symbolic models (which have both linguistic and visuo-spatial aspects). Yes, your models constrain your perception of reality, but it is not clear how important language really is to many of those models (and there is strong evidence it may not matter at all to a lot of cognition [3]).

[1] https://en.wikipedia.org/wiki/Linguistic_relativity

[2] https://plato.stanford.edu/archives/sum2015/entries/relativi...

[3] https://pmc.ncbi.nlm.nih.gov/articles/PMC4874898/

The personality thing seems kind of tautological / uninteresting, as I have pointed out before: https://news.ycombinator.com/item?id=46905692.

Psychological instruments and concepts (like MBTI) are constructed from the semantics of everyday language. Personality models (being based on self-report, and not actual behaviour) are not models of actual personality, but the correlation patterns in the language used to discuss things semantically related to "personality". It would be thus extremely surprising if LLM-output patterns (trained on people's discussions and thinking about personality) would not also result in learning similar correlational patterns (and thus similar patterns of responses when prompted with questions from personality inventories).

The real and more interesting part of the paper is the use of statistical techniques to isolate sub-networks which can then be used to emit outputs more consistent with some desired personality configuration. There is no obvious reason to me that this couldn't be extended to other types of concepts, and it kind reads to me like a way of doing a very cheap, training-free sort of "fine-tuning".

Part of the reason there is too much code to read and review is because we lack the information to contextualize that code.

In many cases, seeing the prompts would help to dramatically speed up rejecting lazy slop PRs (or accepting more careful AI-assisted PRs).

Because "him" is objectively wrong, under almost any interpretation of any words involved. You can cause Claude, or any text-based LLM, to emit language that matches almost any personality / gender / character in the training set. At best you might be able to say "the default outputs have a masculine tone / vibe", but this still doesn't justify, by modern discourse, the "him".

Appreciate this very sane take. The actual code always is more important than the intentions, and this is basically tautological.

When dealing with a particularly subtle / nuanced issue, knowing the intentions is still invaluable, but this is usually rare. How often AI code runs you into these issues is currently unclear, and constantly changing (and how often such issue are actually crucial depends heavily on the domain).

An important consideration somewhat missing in discussion in this thread: if we don't carefully document AI-assisted coding sessions, how can we ever hope to improve our use of AI coding tools?

This applies both to future AI tools and also experts, and experts instructing novices.

To some degree, the lack of documenting AI sessions is also at the core of much of the skepticism toward the value of AI coding in general: there are so many claims of successes / failures, but only a vanishingly small amount of actual detailed receipts.

Automating the documentation of some aspects of the sessions (skills + prompts, at least) is something both AI skeptics and proponents ought to be able to agree on.

EDIT: Heck, if you also automate documenting the time spent prompting and waiting for answers and/or code-gen, this would also go a long way to providing really concrete evidence for / against the various claims of productivity gains.

Try this argument in any field where the stakes are high (e.g. medical diagnostics) and see how far this gets you.

This is really basic decision-theory stuff, often the cost of an error is far greater than the benefit of correctness.

Yup. Realistically there will always be simple changes that AI can handle completely (docs, comments, and commit message), and other changes where some human input will be hugely valuable.

Until then, it makes sense to automatically include some distillation of the AI generation process, by default, IMO.

Something like "it is important to document core / crucial prompts somewhere" covers it. Whether this should be in git or elsewhere is trickier, but doing vibe-coding without documenting any aspect of the process is a recipe for disaster.

Also, how can we (or future AI models) hope to improve if there is only limited and summary documentation of AI usage?

> well, maybe some people do but most people I know re-write the same file multiple times before committing, or rebase/squash intermediate commits into more useful logical commits

Right, agreed on this, we want a distillation, not documentation of every step.

> For example, having some documentation for the intent behind a given series of commits and any assumptions made can be quite valuable in the future, but every single discussion between any two devs on a project as part of the commit history would be so much noise for very little gain. AI prompts and sessions seem to me to fall into that same bucket.

Yes, documenting every single discussion is a waste / too much to process, but I do think prompts at least are pretty crucial relative to sessions. Prompts basically are the core intentions / motivations (skills aside). It is hard to say whether we really want earlier / later prompts, given how much context changes based on the early prompts, but having no info about prompts or sessions is a definite negative in vibe-coding, where review is weak and good documentation, comments, and commit messages are only weakly incentivized.

> Some of these things might be useful to preserve some of the time either in the commit history or along side it

Right, along side is fine to me as well. Just something has to make up for the fact that vibe-coding only appears faster (currently) if you ignore the fact it is weakly-reviewed and almost certainly incurring technical debt. Documenting some basic aspects of the vibe-coding process is the most basic and easy way to reduce these long-term costs.

EDIT: Also, as I said, information about the prompts quickly reveals competence / incompetence, and is crucial for management / business in hiring, promotions, managing token budgets, etc. Oh, and of course, one of the main purposes of code review was to teach. Now, that teaching has to shift toward teaching better prompting and AI use. That gets a lot harder with no documentation of the session!

I mean, sure, a good, detailed commit message is perfectly fine to me in place of the prompts / a session distillation. But I am not holding my breath for vibe-coders to properly review their code and make such a commit message. But, if they, do, great! No need for prompt / session details.

> You have the source code though. That is the "reproducibility" bit you need.

I am talking about reproducing the (perhaps erroneous) logic or thinking or motivations in cases of bugs, not reproducing outputs perfectly. As you said, current LLM models are non-deterministic, so we can't have perfect reproducibility based on the prompts, but, when trying to fix a bug, having the basic prompts we can see if we run into similar issues given a bad prompt. This gives us information about whether the bad / bugged code was just a random spasm, or something reflecting bad / missing logic in the prompt.

> Is it important to know that when the foo file was refactored the developer chose to do it by hand vs letting the IDE do it with an auto-refactor command vs just doing a simple find and replace? Maybe it is for code review purposes, but for "reproducibility" I don't think it is.

I am really using "reproducibility" more abstractly here, and don't mean perfect reproducibility of the same code. I.e. consider this situation: "A developer said AI wrote this code according to these specs and prompt, which, according to all reviewers, shouldn't produce the errors and bad code we are seeing. Let's see if we can indeed reproduce similar code given their specs and prompt". The less evidence we have of the specifics of a session, the less reproducible their generated code is, in this sense.

First N prompts is a good / practical heuristic for something worth storing (whether N = 1 or greater).

I think this is the right analogy, contrary to some other very poor ones in this thread. Yes, it is rare to really look at commit messages, but it can be invaluable in some cases.

With vibe-coding, you risk having no documentation at all for the reasoning (AI comments and tests can be degenerate / useless), but the prompts, at bare minimum, reveal something about the reasoning / motivation.

Whether this needs to be in git or not is a side issue, but there is benefit to having this available.

If you also ensure the AI writes relevant (and correct) docs, and also code comments and commit message, then I agree there is not much need for extra info, e.g. prompts / session distillation. I am not sure that that is the case currently (though we might be getting there soon at least in some cases).

It should be a distillation of the session and/or the prompts, at bare minimum. No, it should not include e.g. research-type questions, but it should include prompts that the user wrote after reading the answers to those research-type questions, and perhaps some distillation of the links / references surfaced during the research.

Prompts probably should be distilled / summarized, especially if they are research-based prompts, but code-gen prompts should probably be saved verbatim.

Reproducibility is a thing, and though perfect reproducibility isn't desirable, something needs to make up for the fact that vibe-coding is highly inscrutable and hard to review. Making the summary of the session too vague / distilled makes it hard to iterate and improve when / if some bad prompts / assumptions are not documented in any way.

Obviously yes, at least if not the prompts in the session, some simple / automated distillation of those prompts. Code generated by AI is already clearly not going to be reviewed as carefully as code produced by humans, and intentions / assumptions will only be documented in AI-generated comments to some limited degree, completely contingent on the prompt(s).

Otherwise, when fixing a bug, you just risk starting from scratch and wasting time using the same prompts and/or assumptions that led to the issue in the first place.

Much of the reason code review was/is worth the time is because it can teach people to improve, and prevent future mistakes. Code review is not really about "correctness", beyond basic issues, because subtle logic errors are in general very hard to spot; that is covered by testing (or, unfortunately, deployment surprises).

With AI, at least as it is currently implemented, there is no learning, as such, so this removes much of the value of code review. But, if the goal is to prevent future mistakes, having some info about the prompts that led to the code at least brings some value back to the review process.

EDIT: Also, from a business standpoint, you still need to select for competent/incompetent prompters/AI users. It is hard to do so when you have no evidence of what the session looked like. Also, how can you teach juniors to improve their vibe-coding if you can't see anything about their sessions?

Nothing other to say than I appreciate you sharing these explicit details and insights here.

Insofar as they are testing very broken and simple "skills" (as evidenced by comments in this thread), and still find net significant net positives in some cases, negatives in others, and huge variability overall, I actually think this is a fun paper providing support that, at least if you count using skills here, you can, in general, very objectively and quantifiably be holding it right/wrong.

I.e. your use of skills resulting in differences of up to ~52 percentage points (or negative percentage points) in improvements (or degradations) in your percentage pass rate is a decent first-pass measure of the importance of skills here.