I think usually you're OK - the scary stuff with lidar is what happens when it snows or it's foggy. Or even worse, a malicious person with a beam that they point into your sensor.
If someone really wants to cause an accident, it's kind of hard to stop them, regardless of whether you're in a self-driving car or not.