Here, since weather certainly changes its fundamental patterns over time, there is no way of reliably predicting out-sample performance.
It's highly likely that in 1-2 years time we'd take all physics-based predictions and all predictions of current AI models, and find the AI model accuracy drops off a cliff when faced with a real out-sample.
The kind of risk with a causal model is that the model itself, of reality, is incorrect. This is a different kind of risk than there being 'no model at all' as in the case of curve-fitting over historical data, which is radically more fragile to expected shifts.
In general, we're quite good at knowing the limitations of causal models, ie., specifying model risk here is much easier. You even, exactly, point out a known problem in the modelling which is impossible to state for an AI model.
Since the AI model is just a weak induction, there are no terms/structures/etc. within that model which can be analysed to understand what parts of it are sensitive to what aspects of the system from which the data was taken.
All we can say is, we know that in general, train/test distributions have to be "nearly exactly the same" for any of these methods to show anything like cross-val levels of accuracy. So we can very confidently predict when we know train/test wont be the same, that this is a mumbojumbo metric.
Indeed, in the vast majority of common ML examples you can, right now, just go and look at real out-sample data collected later than the "common dataset" and you'll find the val accuracy is random or worse-than-random despite arbitarily high cross-val scores.
The dataset which drives me most up-the-wall on this is house price datasets, or pricing datasets in general. Prices, generally, follow geometric brownian motion and nearly all ML models are extremelhy bad at modelling prices. So it's basically pseudoscience whenver anyone uses these datasets to demonstrate anything, esp. predictive accuracy.
Use equations for what we know, use machine learning to fit parameters in the equations, as well as terms we don't know.
Prople nowadays can solve differential equations where some terms are "neural networks" and train those networks on data while numerically solving the equation. (some people call it "neural differential equations", if you want a search query to start)
This seems overly pessimistic.
The weather doesn't change its fundamental patterns over time; those are governed by the fluid dynamics of the atmosphere, and are largely forced (to first order) by things like the amount of solar radiation received by the sun and the the configuration of the continents - neither of which change much at all on timescales of a few thousand or tens of thousands of years. The total manifold of "realistic" weather patterns - especially trajectories of weather patterns conditioned on one or more prior weather sates - is vastly smaller than it would seem at first glance (there's only so much variation in how a Rossby wave can evolve and there is clear asymptotic behavior with regards to amplitude).
I think if you wanted an "ultimate stress test" of an AI weather forecasting system, you could run a a high-fidelity general circulation model equilibrated to the current climate freely for a several thousand year cycle, and randomly draw initial conditions to branch off a 10-day forecast. It shouldn't be _perfect_ (there is likely meaningful skew between the atmospheric state trajectories that arise in this free running simulation compared to reanalysis cycled over the historical observation record for a few decades), but the trends in error growth should be meaningful and directly interpretable in the context of a real-world forecast.
ML models are models of measures, not of the causal process which generates those measures. In this sense they arent 'models' at all, in the traditional scientific sense of the term.
> The model was trained on four decades of temperature, wind speed and air pressure data from 1979 to 2018
The test set is taken from after the training set. I agree it seems likely that it will need to be continuously trained on new data to maintain it's accuracy going forwards, but this seems like a promising sign that it should work as long as you keep updating the model.
Of course the "can compute weather in 8 minutes" thing doesn't include the ongoing training cost... so the "this is much cheaper to run" claim seems slightly suspect.
You can get a lot of traction with your assumptions but it's that 2-3% of predictions where being accurate matters a lot.
i'd never heard of a storm doing that, but i am not really that in to weather, so it's possible it's in the training data somewhere?