Add to this complexity the weather conditions. Suppose the sun is shining straight at you and you need to squint and shade your eyes just to make out what the light is -- this happens to me frequently -- can the camera see the traffic light and distinguish its color clearly under such conditions?
What about when it's raining, misting or drizzling, snowing heavily, etc. and the traffic lights are these fragmented outlines that you, the human, can heuristically distinguish but a machine might not?
One last thought: suppose it's right turn on red and first car in line is a self-driving vehicle. Can it really look left and safely determine there's enough time to beat the cross traffic? If it's highly conservative and just waits until green, there could be ten irate motorists behind it and guaranteed to honk and curse.
It's exciting technology but there are some very difficult problems to solve. I worry that if these machines can't demonstrate 110% of a human's ability to drive, they simply won't be implemented in many places except some very well defined rigid routes that are free of problematical challenges and variations.