I would prefer if we could have more certainty than that. I have a list of questions.
1) Does no one else have access to this codex? Has anyone tried to replicate this? It should be easy.
2) If it was not fine-tuned, there is still the issue of an over-fit question set, as you say. How easy is it to break? How general is it?
3) Are there really so many extensive examples on the use of sympy? Some of my googling (I know the math but not sympy) did not support this but it could be my lack of familiarity of the library ecosystem.
4) And the biggest issue where when it gave answers to certain limit problems without any computation, which should not have been possible unless it computed it internally, complexity of which kinda contradicts the entire exercise.
Depending on how generalizable this result is, it could be anything from merely impressive to mind blowing and revolutionary. Imagine if the language is Coq or Lean instead of sympy!