Instead we have to understand the distribution, and the mean as well, and add up how long it takes on average, rather than just in the most numerous case.
Further, and perhaps more importantly, by understanding each distribution, you have far greater knowledge about what the expected time will be, and where the delays are most probable; and you can begin to start analyzing where those delays might be coming from in the interactions of all the parts of the system. This is straight-up quality talk right out of W. Edwards Deming, and it's how you start to improve how you produce in general. Good stuff.
Maybe optimistic estimates are a motivating factor for teams. The desire to finish faster, to be more efficient than you were before, a commitment with a challenge.
In my experience, it takes a strong product owner and a mature development team to meet deadlines. Shipping on-time is always a game of tradeoffs. Accuracy in estimates is usually the result of doing things in a known, measurable way. And by keeping the stakeholders close, you can make critical decisions together to keep a project on track.