Ask HN: Is it possible to backtest math/scinse breakthroughs with AI models?

I'm not sure what you expect to find. If there is a better theory, you can still publish it now.

There is a problem with classification and arbitrary choices like when to add a leap second or if Eris is a planet or how many colors the rainbow has.

It's been attempted with real data: https://github.com/haykgrigo3/TimeCapsuleLLM

However, it is necessarily a tiny model because of the limited training data.

All machine learning is extremely data-intensive compared to organic brains, it's nowhere near good enough to generate novel (relative to the training cut-off) maths or scientific hypotheses.

Yeah, like plan out every known tech, and then try to duplicate their every finding, and maybe simulate the gossip of the intellectual culture at the time from journals and open letter publications.

Sounds like your setup would take ten years.

How about building a “tech tree” of known significant progressions and work it from there!