On the flip side, like you suggest, the cost of being wrong can much higher than many current uses of AI. This is where research into AI interpretability, robustness, and uncertainty quantification can really help.
On the flip side, like you suggest, the cost of being wrong can much higher than many current uses of AI. This is where research into AI interpretability, robustness, and uncertainty quantification can really help.
In the case of reducing compute demand where you might want to find efficient shortcuts to run high fidelity simulations faster, one could use these to create a candidate space of cases to then run the actual simulations against. This may still reduce the total space of simulations you need to run. Lots of potential here without having to make too many sacrifices.
The real issue we have in the simulation space though is overpromising. People give far too much credit to some of these models and simulations that, while sometimes rigorous, often don't fully capture the phenomena they intend to. At the end of the day, even the scientific and engineering simulations are just another pass/filter to narrow down the cases of experiments you want to try in reality.