> We know that transformers can generalize within the training set.

> Of course it can't generalize beyond training - why would it?

4 out of 5 people I discussed this subject didn't know, and even believe that current LLMs are bound within their training set. They claimed that LLMs could synthesize data beyond their training set, and the resulting answers will never be wrong.

There's a large misunderstanding about how these things work, and LLM developers do not spend the effort to fix this misunderstanding since it helps to raise the hype even further.