"Consequently they cannot generate and recognize matching brackets correctly."
Can you expand on this? Not quite following.
Can you expand on this? Not quite following.
I'm not sure how much you know about regular grammars, but basically they're the kind of thing that a regular expression can match. Now regular expressions can do a lot but they have their limitations, in particular they cannot distinguish a sequence of matched brackets '((())())' form one of unmatched brackets '(()(', or at least not with 100% accuracy.
It turns out that regular expressions are precisely the languages that can be recognized by finite state machines. Which is kind of equivalent to the possible outputs of a markov model.
Since large language models only have a finite number of states they must have the same limitations, which means it is fundamentally impossible to make them only generate balanced brackets. They get away with it by having a ridiculous number of states, so they might not be able to deal with arbitrarily deep nesting, but they can still get far enough that you won't generally notice.
ChatGPT Example:
> Add the correct number of closing parentheses to this string: (((((((((((((((((((((((((((((((
>> ))))))))))))))))))))))))))))
>> The correct number of closing parentheses to balance the opening parentheses is 21.
which is not correct
I think an analogy would be like saying that you can't represent pi as a floating point number. Precision can be increased by adding more bits, but there's a fundamental limitation because of the underlying storage mechanism.