It makes me think that the first real wide scale self driving vehicle deployment will have to come from a nation state rather than a corporation; the easiest way to solve all of these problems is by putting heavy investments into smart road infrastructure such that the car depends on sensors embedded in the signs or the asphalt rather than any camera feed. A school bus could just have a transmitter to indicate whether its lights are on or not; construction workers could be given road signs that transmit similar signals or better yet use a portal to construct a virtual geofence around wherever the construction is happening before it begins. These are all easily solvable problems in today's world; what is missing is the political capital to invest in these solutions despite the insane ROI.
This infrastructure will need distributed two-phase commit. For safety-critical issues like school buses, eventual consensus won't be good enough. And then you'll need an infrastructure that detects when an older or simply defective car is present but not participating in the protocol. These problems are all solvable, but they're by no means trivial.
That is exactly why they can't interact with real world cases of fully valid plastic signs coming out of nowhere.