Maybe post-LLMs mathematicians will work on problems/topics that even ChatGPT can't solve? Like, P=NP remains unsolved, we can work on that. Sure, it's possible that it will get solved tomorrow by an AI or somebody else, but that possibility was always there.. Or, to be more practical, one of the claimed OpenAI results is that integer multiplication is sub n×log(n), so assuming it stands, now there's a new question: "What is the actual lower bound for integer multiplication?"
I have no proof, but I believe (i) LLM progress will converge at some point, (ii) that point will not be so far off from human intelligence, since they're trained on human intelligence, so.. not all problems will be solved/solvable by LLMs. But they'll be fast(er), like calculators are faster than us.