This isn't "physics in this way" - it's a computation and machine learning problem that still has not been solved with purely visual sensors.
So maybe they shouldn't use purely visual sensors until the problem is solved? It's not like they don't have any other option.