The problem is when the car you're going to narrowly miss is still in the path of your car but going to move out of it. That's something a human could predict based on judgement, but it's very hard for a basic collision avoidance algorithm to predict.
Humans also get this wrong. You see a car turning, you assume they keep turning and you'll pass just behind their rear bumper in that case. Now for a reason invisible to you the turning car hits their brakes and stops partially blocking your lane. You can't stop in the remaining distance. But would usually be able to steer around it by going to the edge of your lane or overlapping the next lane. But this isn't something a simple collision avoidance algorithm can do.