LLMs are pattern prediction systems with a large training data set. It is not surprising that they can predict patterns, particularly for a well structured field like mathematics that is also amenable to automated proof checking to help steer it.
What part of my statement do you take issue with: that LLMs are pattern predictors (that's literally what the algorithm that runs it does) or that mathematics is rules-based and checkable and therefore amenable to automated pattern prediction?
If you are able to check that the results of an LLM are satisfactory, it means that the results are checkable. Then, in retrospect, the process of LLM coming up with those results is rule-based, because an LLM is a large set of data manipulation rules.
In short, which concrete thing that an LLM does would surprise you?