Hacker News Reader

Many neurons are polysynthactic, that makes interventions like the proposed difficult.

Is that necessarily the case for massive LLMs, or could there be a more refined grouping?

I am not sure if the origin of polysyntacicity is fully understood.

From a physics perspective it’s entropy: There are just more local minima that have neurons code multiple things.

I suspect that dropout and similar tricks also play a role in this: Removing connections during training means that pathways need to be redundant somewhat.

That's super interesting, I wonder if dropout applied in a certain pattern of segregation could induce more neuron dependence, instead of less. Or do some anti-dropout where a dedicated portion of the model is used. Not sure why you'd want that, but it might be interesting to explore.

Reply on news.ycombinator.com