Are you sure about that? It looks like artificial neural networks can, in principle, do anything biological neural networks can do.
I think it's more likely the opposite: there are steps we could never reach without computer assist.
Isn't open ended research is truly open ended. Like there are a gazillion questions to ask and even more solutions that can be offered. Now which questions to pursue is human volition. Yes, humans will not be able to reach certain steps without computer assist.
However, which questions to ask is a human endeavor, out of infinite possible pool of questions.,
now computers can ask questions as well. But if it were to ask it will be more like a monkeys and typewriters type of a situation. Tons of questions and a few that are relevant for the interested human., who has to foot the bill for those questions.
Surely an AI could consider the human's needs, wants, and interests, and then provide a likely candidate for a good question.
The question for mathematicians is: what can a mathematician do with these new tools that no one else can?
Waxing nostalgic about the sound of pencil on paper or the beauty of your abacas is a different conversation, as is the complaining about who moved my math cheese.
This is simply a convenience for power efficiency, and so they don't rise up and murder us all. There is not a single law of physics or the natural sciences that you can list here that says we cannot grant agency to a generalized machine and allow it to become a meta-optimizer.
If in the exercise of your agency you create or release a generalized machine and allow it to become a meta-optimizer, then you are responsible for it, because it has not agency.