Probably right. Most people dump on these language models for this reason but it would be absurd for a HS student to have to re-derive the quadratic equation every time they worked on an Algebra problem so naturally you memorize it. Why should it be any different for a language model?