D-Machine

Scanning back, I can see I was responding here was what I was worried was a mistaken assumption that a meaningful abstract / summary can always be written for truly complex and novel topics. Sometimes all it can be is a sales pitch for reading the details, and I thought the summaries here were good enough in this sense.

But I agree the author is obfuscating weak claims via length / complexity (and it may be as much for his own sake as for his readers'). The basic claim ('maybe AI will be more explainable/tractable than initially thought') is IMO intelligible, and not completely nonsensical, but also extremely vague and vibesy, which, technically, I must agree is in the same territory as nonsense, at least.

Based on our other exchanges here we mostly agree with the problems of the article I think.

> Neurons having narrow semantics means you can inspect what a neural network is "thinking" with confidence

Not really, because of emergent behaviours and also because there are still millions of neurons being summed (with variable weighting depending on inputs) at each layer.

> and even constrain its space of behaviors by activating/suppressing a small set of neurons that have very straightforward effects on its behavior.

You can influence behaviour by doing this kind of thing, yes, but it isn't straightforward at all, and is generally very messy and you get all sorts of unpleasant sides effects (e.g. abliterated models not performing as well).

Yes, "alignment" is very wishy-washy and often does mean pointless moralizing / following of social norms. Obviously thinking about alignment in that sense will be a waste of time, since we don't have particularly consistent or universally applicable ethics (or frankly even general knowledge) in the first place.

But if the term is understood in a more narrow and applied sense, it is a useful shorthand (or can be). Jon Stokes said it nicely on Twitter recently:

    There is no "alignment problem" to actually "solve" in 2026. The "alignment problem" presumed a single, stable utility function in an agent, and that's not what we got, so people should stop pretending that's what we got or are going to get.
    
    In place of "the alignment problem" we have a family of steering and control problems that cannot be separated from the specifics of an application context. We have process quality & control problems, like the engineers who work on nuclear reactors or chemical processes.

https://xxcancel.com/jon_stokes/status/2110404230053802443#m

There is indeed a huge amount of useless jargon and IMO deliberate obscurantism in deep learning research.

Sometimes it is to defensively ward off critics, sometimes it is to woo the naive and ignorant, and sometimes it is because the basic ideas are really just incredibly simple and based on simple mathematical intuitions or hunches.

In the latter case, papers and authors appeal to some obscure or overly-complicated technical theory (which usually doesn't actually apply at all to current deep models as they are actually implemented, e.g. infinite-width networks and other such purely theoretical abstractions) to cover up the fact that they really just had a lucky intuitive guess (and likely that twenty other intuitive guesses just didn't pan out). This appeal to theory is often self-deception as much as it is appealing to academic reviewers.

I think this article is a bit of Nick wooing himself and others here... I'll wait for his supposedly forthcoming paper to see if there is any real meat in it.

Yes, that comment is much better and appreciated. I agree post-hoc description is not the same thing as mechanistic explainability at all. The article is indeed flawed in sort of muddying the distinction between these two.

EDIT: Actually the article makes the distinction at one point, but the emotional / hopeful thrust of the article relies heavily on an assumption that understandable post-hoc descriptions of e.g. neurons and circuits (model components) are going to help lead to real mechanistic explanations and more specific model design.

This has not so clearly been the case in other fields (e.g. neuroscience, human psychology, economics, etc), because the problem of emergentism / the whole generally being much more than the sum of its parts / components.

This was one of my intuitive concerns reading this post.

I've done work with trying to train explainable models for e.g. making diagnoses based on neuroimaging, and it is not clear to me that if you applied these techniques to e.g. vision models trained exclusively on such brain images, that these visual representations or conceptual clusters and relations would really help understanding at all.

The main issue would be that you'd have no clear way to verify any generated explanation for such training data. I.e. we can "verify" some of the explainability shown in the article because clear and recognizable image clusters and/or semantic groupings pop out, because we already understand the domains. But if this were fMRI or e.g. EEG signals, we really wouldn't have that, so the explanations would themselves just be more hypotheses.

Perhaps if we can build on interpretability of vision and language models in domains we understand, we can eventually generalize this to other domains, but I don't think that clearly follows from anything I am reading here.

GP and the article both make clear arguments for WHY they believe this. You just asked a lazy drive-by question without providing any reasoning at all for your position. If you expect a good response, you need to provide something meaningful to respond to, as it is it just looks like you are providing knee-jerk skepticism without having engaged with the material.

> There have been large multitudes of articles by others who are arguing the opposite

Also, Nick's entire article is framed such that, yes, plenty of others have argued this, and even HE used to believe the opposite, but now does not. So your point about other articles here seems to me to be a sort of non-sequitur.

What theory says this? Surely not a proven or applicable one, as this post makes it very clear a great deal of understanding is in fact possible, even with highly complex neural networks.

Unfortunately not all ideas can be meaningfully compressed into just a few sentences without completely destroying the important parts, and this is especially the case in higher-level math, stats, ML, and deep learning.

Also, IMO there is an effective abstract anyway under the title text:

    The cause of polysemanticity in early model organisms wasn’t superposition, but several ordinary, mostly fixable issues. We can now understand most neurons in models up to trillions of parameters, and think the outlook for mechanistic interpretability may be far better than we thought.

But you are right the article needs significant editing down.

The posted article argues this many times...

Using "orthogonal" here would have the opposite effect, as it would impart a far more specific meaning which is also incorrect in this case.

Article is pure bullshit (i.e. doesn't care about truth, just wants to sell you something). You can point any competent AI at it and see how selective it is being (or just know it is BS by being informed, e.g. https://www.sciencedirect.com/science/article/pii/S088915751...). That this gets upvoted on HN is concerning and suggests a serious bot problem, this trash should never persist on the front page.

The concerns about meaningful decline are also essentially fake / bullshit https://www.sciencedirect.com/science/article/pii/S088915751....

EDIT: Also being worried about nutrient deficiency on a modern balanced Western diet is also basically deranged and wildly out of touch with the evidence. There are some interesting exceptions for women and iron, to some degree vitamin C (easily solved), and then some serious B12 issues in vegans, but, otherwise, vitamin deficiency is basically a non-problem in the developed world.

Sigh, more sensationalist, low-quality coverage, and, as always, proper scientific studies refute these claims: https://www.sciencedirect.com/science/article/pii/S088915751....

Also Deep Science Ventures / Hollow Harvests is a biased startup that simply cannot be seen to be objective on these matters.

Right, so it is broadly seen as more on the left than the right. Or, to be glib: only a radical leftist would think NPR doesn't lean left is far more correct than your initial assertion.

Try thinking for just two seconds about what your process would result in, given the scale and amount of results here. You are basically asking for these results to become available to all experts only months / years from now, only after some select experts are chosen and first get to read these results. If there is any error or bias in that selection process, the whole thing is delayed even further or lost to the file drawer entirely. Or, since you say these are all unreadable, and should be rejected, you are in fact arguing these results should never be released at all.

Which is why it is obvious post-publication peer review is the only sane way to handle this. Formal peer review is not even remotely up to the task here.

No. Letting some tiny handful of individuals selected by a tiny set of review-delegators review this is immeasurably worse than just releasing it publicly and letting anyone with the sufficient expertise review it. You know nothing of how peer review actually works, or have not thought about why that process would be only harmful in this case.

Not true at all? https://ground.news/interest/npr

Why do people still trot out this tired argument?

https://web.archive.org/web/20230404013504/https://freddiede...

EDIT: And if you didn't mean it in that way, and were serious, consider looking at e.g. Ground News' rating of NPR: https://ground.news/interest/npr. They (and others) clearly show NPR to have (slight) left bias. This means you are almost certainly going to see woke perspectives there from time to time, and more often than other less-biased or right-biased sources.

CAR-T is not a cure in any sense of the word, in part because "cure" is just not a scientifically valid concept in oncology. I would know, CAR-T saved my life, but this was at great cost and is not even remotely close to a guarantee, and the side-effects can be beyond devastating https://news.ycombinator.com/item?id=49827669. At best, CAR-T is more like a tradeoff: often just a coin-flip's chance to live, for long-term—maybe even permanent—life-shattering consequences.

Correct. The people here saying CAR-T is a cure have no clue at all what they are talking about, and I say this as someone that is only alive right now because of CAR-T https://news.ycombinator.com/item?id=49827669

There is no world in which CAR-T is a "cure", especially since this isn't even a scientific term in oncology. We generally say if there is no relapse after 5 years post-treatment, then any cancer is a "new" cancer, so 5 years of remission is the closest thing to a "cure", but this isn't a scientific term.

Also, we barely have more than 3 years data for CAR-T for most cancers. And even still, the survival rates aren't great, in many cases 50% compared to e.g. ~20% for previous chemo-immunotherapies plus marrow transplants. And this ignores how massively immunocompromised (or so permanently brain-damaged you are effectively senile) CAR-T can leave you. You can be severely immunocompromised (literally identical to or worse than AIDS / late-stage HIV) for at least a year in close to half of cases, but maybe even permanently, in perhaps as high as 10% of cases (at least for lymphomas).

I say this as a person that is only alive because of CAR-T treatment 1.5 years ago. CAR-T is amazing, and a far better treatment than previous treatments, but calling it a "cure" is deeply misleading and mostly clueless. Currently, it is simply a much better last-ditch effort than the previous ones.

HN is now heavily astroturfed or overrun by bots (or, alternately, low-quality posters now indistinguishable from the previous), and this is especially so in AI-related threads.

One tell is that most comments barely exceed one or two sentences (because otherwise AI detection gets easier and much more reliable), when this was not as much the case many years ago. The drive-by comments are also low / zero quality, mostly expressing a feeling or agreement/disagreement, and primarily driven by ideology or pre-existing beliefs and commitments.

Look at non-AI-related threads and you'll notice a large distribution shift relative to AI-related ones.

EDIT: Basically HN is orange Plebbit now. If you doubt this, compare HN discussions to those on e.g. lobste.rs, LessWrong, The Motte, DSL, ACX, or other old obscure forums. You'll notice those places have their own very serious biases and problems, but it is obvious the vast majority of posters are nevertheless human and making some minimal efforts.

Now compare Reddit and 2026 HN to the above, and see if you can confidently say the same.

Falling into deranged ideological projection is also an interesting choice - one I chose to mostly ignore. You seem to really obsess a lot about this Yarvin fellow: I tried reading his stuff once and found it intolerable.

I imagine you think calling some of my language choice a "slur" here is some kind of gotcha, when the term I used is specifically one widely disputed as actually being offensive, given it is mostly used now to refer to normal people acting in intellectually deficient ways, and not generally to those with actual learning disabilities that deserve our sympathy. There are studies on this, which you surely are aware of.

If I had referred to your more deranged positions as "smooth-brained halfwit extremes", you likely wouldn't haven't tried to impotently pull this "slur" card, even though the semantics are basically identical. Which basically goes to show that you value irrelevant surfaces over substantial realities, and frankly is perfectly consistent with the midwit intelligence denialism on display in your posts in this exchange.

> Cognitive disparity is because of compounding investment of attention and metacognition in development [...]

Sure, but exclusively? There are no other factors that don't depend on effort / investment / social context?

I can't take you seriously when you take such an absolutist stance on these things when science has long since accepted nothing complex about humans is 100% nature or 100% nurture (really, shared vs. non-shared environment vs. genetics: but, surely you know this).

"Abstraction ceiling" is just a way to talk about intelligence at at the tails that reveals one of the difficulties / limitations you can encounter when it comes to compressing / abstracting complex mathematical objects. Your objections to the term are facile and clearly stem from an obviously unvocalized intelligence denialism that is simply indefensible today. Also, intelligence != IQ, obviously this is too simplistic.

Everything about your arguments and other posts is similar reductions to retarded extremes (our only options are "eugenics 2.0" or deranged intelligence denialism - there is no room for anything in between, e.g. the idea that base intelligence matters and sets a hard average ceiling on potential, but that effort and other factors might push one slightly above/below this ceiling relative to others with a similar intelligence, and etc). Or alternately you hallucinate things I never said or even remotely implied (e.g. we should gatekeep based on dumb psychology metrics or hand sizes).

Just be honest: you know intelligence is real and matters, but you want to dance around this fact because you find it ideologically inconvenient, or you can't admit you yourself have limits (and lack the courage to realize the obvious social broader consequences of this personal admission).

> But everyone is as a kid when they learn language and everything else

Also clearly false by almost all current research.

> So the task is just to get people back into that mindset.

Again, you have no evidence, and this is clearly wrong in cases of mental retardation or brain damage. Modern genetic studies also seem to suggest intelligence is related to a lucky absence of errors / genetic problems that are otherwise inconsequential (or even advantageous) in other domains, so really, your "everyone starts perfectly equal in intellectual ability" is just empirically disconnected and ignorant fantasy.

Uh huh. A bunch of quasi-mystical bullshit to justify utterly inept garden variety intelligence denialism (it also just shifts terminology: if we accept your metaphysics, it would still be strange to propose that everyone has exactly equal "hacking" ability).

The hubris and willful ignorance required to imagine that everyone is just equally and infinitely unbounded in their cognitive ability is simply mind-boggling in 2026.

IMO this would track, applied mathematics (even e.g. data science, though perhaps calling that applied math is a bit generous / insulting to more serious applied math) is in some ways more exciting now because it is far easier to surface complex / appropriate methods for the task at hand, and you can more confidently explore these methods because the AI sort of "has your back" in catching some of the more obvious beginner errors you make during these explorations. Plus, applied math feels roughly more results- than process-focused, compared to pure math.

IMO the divide here between pure vs. applied math feels a lot like the divide between those who enjoyed coding for the understanding it led to, i.e. the writing itself was the joy, vs. those that primarily coded for the results. I enjoy the creative part of coding, the thought of software jobs just devolving into writing specifications and doing code review very much kills it for me.

Find me the magical teaching method that can make anyone learn any kind of mathematics at nearly the same rate, and you have falsified the idea that anyone has mathematical limits.

Given we as a society can't even figure out how to do this for educating stuff involving simple fractions, my theory is far superior than whatever exactly it is you think.