When AI has exhausted the existing math literature (and found a bunch of errors, I'm sure) it will have to start producing its own conjectures. Millions of them, even billions. And using all the experience from trying to prove them to improve.
Insight in mathematics doesn't emerge from brute-forcing conjectures. There is a recognized aesthetic economy that exists and without a semantic filter to detect the significance in any of these proofs we just end up with a lot of noise.
Yes, it would be very much like automatically generated tests. These are not worth much by themselves, but in massive bulk they can do interesting things.