There is no current mechanism for updating something like ImageNet. There is no place you go to point out problems or contribute changes to keep it in line.
We really need to make datasets social. This has a whole host of challenges like copyright, ownership, versioning, and even hosting costs. Even with all that it is a tractable problem though.
It's in fact the data itself is the closest to reality, not the ML model...
It's a conundrum in the flow of reality for me:
Reality -> data that reflects reality (hopefully as best it can) -> train model based on data (typically takes a non-trivial time scale!) -> make decision / do stuff based on algo with newest data
If you ask me, even with all known technology we have, seems like it's impossible to simultaneously aquire realtime data and train a model to operate and or make decisions on exactly that data. It's a catch 22, there will always be a lag
Even as humans with super big brains we can't hope to do this outside of extremely simple tasks like "throw and catch the ball"