Well, I don't know about mathematicians. I'm a computer scientist. I study computers. And I know that the behaviour of a computer is entirely determined by its design, and its inputs. If a computer is doing something that is not in its design to do, then that's because there is an error in the implementation of its design. And if it's difficult, or impossible, to anticipate the behaviour of a computer, that's because it's programmed to do something unpredictable- like in my RNG example above.
Computers are machines. Programs are other machines, running on computers. There are no spirits in any level of a computer's functionality, no magick, and no metaphysical mysteries to solve.
Maybe that's not how it works in other sciences, that study the natural world, where there are too many unknowns. But in computers there are no unknowns. We make them, and we know how they work.
Regarding LLLMs, we know why they're so good at what they do, because we know what they do, because we are the ones who made them to do it, and we know that what they do gets better the larger the number of parameters and the larger the data they're trained on. We've known that for decades.
And as to "emergent behaviour", that is just asking for a mystery to be there, when no mystery is anywhere near the whole thing. "Emergent behaviour" just means what I say above, that there's a complex system that displays behaviour that is difficult to predict, even if its design principles are very well known.
There's a time to wonder, and wonder is the seed of science. But to wonder about a machine that is doing what it's designed to do, is not the wonder that leads to science. That kind of wonder, well, it reminds me of the old joke:
They ask a mystic what is the greatest mystery in nature.
"The thermos!" he says. "It keeps my tea hot in the winter, but cold in the summer. That little bottle! How does it know?"