How robust will ML models be in the world of rapid climate change, where the past no longer predicts the future?
Weather predictions are for specific events and areas, made on the order of days- typically no more than 2-3 weeks into the future.
Climate models predict future averages over large regions on the order of decades.
"rapid climate change" is on the order of "within this century". Whether the climate changes or not doesn't really impact the weather models at all, because the climate's not changing on the same time scale.
Weather prediction: which path typhoons will follow?
Weather models look at current conditions and extrapolate what the next set of conditions will be.
Another way to look at it is that climate change models reflect what we will experience. Weather models reflect the mechanics of humidity, temperature and air flow- given a current state, what is the next likely state. Climate change doesn't change how those mechanics interact. It only guesses what "current state" will look like in the far future.
Over the course of ten years, the average daily temperature goes up by 1 degree F (which is many times faster than reality).
That means that there's basically one day in the entire year that doesn't fit within the existing model, or maybe a handful if some other part of the year is cooler than normal. We see more variation between normal years than we expect to see average increase in both the short and medium terms.
Our weather models predict no more than two or three weeks into the future.
There's literally nothing about climate change that will change how weather systems interact that will occur faster than our weather models will adapt to. Our weather patterns may change drastically, but that's a fundamentally different problem.
Those models don't just "adapt", they're carefully and manually refined over time, slowly incorporating better physical models as they're developed, and better methods of collecting data as it become available. Climate change absolutely changes how weather systems interact faster than our models can adapt, primarily because we can only adapt models to changes after their accuracy degrades, and it becomes possible to start identifying potential weaknesses in the existing approaches.
You're example of seasonal temperature changes vs global average temperature increases, vastly underestimates how complicated weather models are, and how much impact changes in global climate systems (like the North Atlantic Current) can drastically impact local weather conditions. Weather models, like all models, will make assumptions about the behaviours of extremely large systems which aren't expected to change very much, or are very hard to measure in real-time. As those large systems change due to climate change, the assumptions made by weather models will grow increasingly incorrect. But that hard part is figuring our which assumption is now incorrect, not always easy to identity exactly what assumptions have been made, or accurately measure the difference between the assumption and reality.
Thus, if you want your AI model that predicts weather, which was trained on data before that change, to predict weather after that change, you may very well find that it rapidly loses accuracy.
And every year over year change is within error bars of the prior years results, once you account for those initial conditions.
But if your model is an LLM (or close cousin) purely based on previous weather patterns, then it may struggle to accurately predict once the underlying conditions change.
If you train a weather model ML on past weather data, which has reduced variability, and use it to predict future weather, it is possible that it under predicts variability of weather variables (amount of expected rainfall, maximum temperature tomorrow etc). Variance not mean.
I say "it is possible" because only rigorous modeling can tell us the truth. Which is why I asked the question originally. I don't think it can be denied by abstract arguments.
I can hear the keyboard mashing already. YES, I am aware there are problems in both research and publishing.
That does not mean that a HN'ers "did they think of X?" is any more valid. It's like saying that because bridges collapse, we should be allowed people with no relevant experience, knowledge, or training to look at new bridges and say "that bit there doesn't look strong enough to me!"
The GP asked _how_ robust the ML models will be, which is a perfectly good question to ask. Maybe a climate scientist specializing in ML can answer that question.
"It's like saying that because bridges collapse, we should be allowed people with no relevant experience, knowledge, or training to look at new bridges and say "that bit there doesn't look strong enough to me!""
Not a valid comparison. Civil engineering is a field with a lot of known answers. bridge designs rarely rely on cutting edge research.
I have heard the opinion that the models would not learn weather patterns but rather “weather physics”. I am not knowledgeable enough in ML to comment on that though.