Haven't I seen lots of stuff showing LLMs do arithmetic for examples they haven't seen? If there is an issue really grasping basic logic though, doesn't that put a damper on the spooky "emergent properties" explanation for stuff like addition?
No.
The absence of one desired emergent property isn’t evidence against a different, observed property. (And “emergent properties” isn’t an explanation as much as a statement that we don’t understand the mechanism by which the training data encodes the knowledge and did not plan for it to be encoded.)
Sure, but why one and not the other? You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic. I don't see an obvious reason why rules for general arithmetic would just fall right out and a few basic rules of logical inference are much harder to get to.
If we could describe the interaction of factors that determined it, we wouldn't call it an emergent property.
> You'd think there would be more examples of simple logical relationships in almost any training set than there would be examples of arithmetic.
I'd think the examples of arithmetic relationships would be more consistent (and often symbolic), while given the ambiguity of the language relevant to the kind of logical relationships at issue, it would be more murky to even infer the broad rule, and less clear from the language of particular examples whether the broad rule, even if it could be inferred, applies.
> I don't see an obvious reason why rules for general arithmetic would just fall right out
Again, if anyone could explain an obvious (or even non-obvious) reason for that, it wouldn’t be described as an emergent property. It would be described as a function of the specific things that contribute to it.
I take your point about representational differences though.
Thinking more about this, the OP is basically talking about whether models understand that equality commutes, and I suppose even an LLM that can add may not have worked out that addition commutes.
the reality is that the emperor has no clothes (cant compute). LLM is a great assistant but is not showing any sparks of AGI. Those seeing sparks of AGI are in denial due to vested interests.