> You can maybe identify "traffic light", "car", "pedestrian", and "deer" with machine learning.

What you wrote makes a lot of sense to me. However, I would also hope that in a matured self-driving environment, traffic lights would be mapped ahead of time. It's always possible that a traffic light shows up in an unexpected place, but in my mental image of the future, this raises a flag for system management to modify the map of permanent objects.

There is a chance that a powered non-permanent traffic light is found somewhere, a dude walking along the side of the street with a stoplight on his back, and connected batteries. In that case, the police need calling. In machine learning, there are just so many corner cases where best to not try to fully interpret, just call out "dude, look at this thing!"