The only way a LLM can come up with new ideas if the "idea" appeared as a generalisation durring training or if it was achieved using reason in chain of thought.
Ordinarily in mathematics there’s a TON of papers to write just combining low level problems with different techniques. Better still, and often considered groundbreaking is borrowing techniques from other fields and adapting them or creating analogous methods to solve problems. A lot of landmark papers have been written this way. This is also what transformers are sort of good at within other contexts. They have super human breadth so I’m hopeful they’ll become real assets in math for a long time. Though the leaps necessary to adapt a technique in a nonobvious way might be too much for a while longer. We’ll see.
Truly novel techniques are quite rate indeed and I don’t know if LLMs can represent them faithfully in their embedding space or not. My inclination is that they probably can most of the time, but I don’t know. Mathematicians would describe such thins as “alien”.
A jump in intuition comes from automatic processes reorganising the relational structure of conceptual models. There is no reorganisation of the model durring inference.
one could argue that model can reorganize / interact with prior knowledge captured in text form (edit files) hence there can be reorganisation