The latter is a problem with LIDAR as well. In any system, you're going to need sensor fusion to get a robust result.
The problem with camera based systems is that they don't know that they are missing data, and the system just thinks the path is clear...