Our weird behavior during the pandemic is messing with AI models
technologyreview.com
technologyreview.com
This has prevented us from the pain of having “bad” unreliable predictions which are worse than having “no” predictions. This saved us a lot of pain in maintainence and debugging.
[1] https://en.wikipedia.org/wiki/Structural_break [2] https://en.wikipedia.org/wiki/Simpson%27s_paradox [3] https://slatestarcodex.com/2017/09/06/predictive-processing-...
I always find myself falling back to hand-crafted heuristics + Snorkel [1]. Is there anything better?
Wave vs particle, gradual heating vs phase change, but also soup with little chunks of meat (themselves smaller continuum), gas clouds coalescing into planets. It goes on and on.
In this case model was built for the continuum and in came a big discrete shock. Something it wasn't built for. Shocks require a different modality of thought. More reasoning, logic-y than regression.
Elements of it show up in autonomous vehicles. For example, one thing that can cause a disconnect is when sensors detect a sequence of events that is extremely improbable. If it's improbable enough, it's probably not the world, it's probably your model. Best to hand it to a human or hit the brakes.
It reminds me a bit of John Meriwhether's statement that the events that led to LTCM's collapse were a "Six Sigma Event." If you are getting a six sigma event and you've only been in business a few years, maybe it's because your probability distribution of the market is wrong when you get a little too far from the mean.
Time to broaden the scope again to include cybernetics, reasoning, counter-factuals and so on rather than just building end to end data driven learning systems.
There has been some work on self-training or continuous learning AI, where all new inputs are used to continuously retrain the model, but these can be difficult to control, and training is often the most computationally expensive part of AI, so it often is not feasible except on arrays of computers. For example if people realize the model "learns" they could change their behavior to make the system learn to do the opposite of what most people would expect, not a very desirable outcome. Or the "learning" might home in on some strange artifact of behavior and accept it as the preferred output, such as thinking that when people click their tongues they are saying "yes", when really it just happened that some group of people who happen to do both were using the system a lot for some reason. This is still an area of research and while progress is being made it's much more difficult than using a large collection of data to train a model. For some kinds of problems the "learning" needs to understand context, which is very difficult to do in software since it involves understanding how the world works.