Ask HN: Why do people choose 'over-confident' over 'cautious' – how to stop it?

So many times in my career have I seen situations where given a choice between people who are 'sensible and cautious' vs. 'over-confident and dangerous' people choose the person who is offering the later. Then they will even double down on their bad decision with various confirmation bias' even when things go south they end up thinking the 'over-confident' people were great.

I remember one project, where a director in a company I worked at, had a choice between a vendor who was offering a realistic timeline at a good price OR a vendor offering a totally unrealistic deadline at a very low cost. He choose the later and obviously the project was delayed and delayed and delayed, cost x10 the original budget and caused insane stress and damage to relationships, people quit etc. After the debacle was over I was talking to the director and he was still happy he'd chosen the second vendor and wanted to work with them again!

Another common pattern here is when these over-confident people keep jumping into projects, forcing bad decisions on teams without understanding any context and then parachuting out before shit hits the fan.

What's going on here from a psychological point of view? Is this an optimisation - are people choosing 'over-confident' people because they think they will need to make less decisions and someone else will take responsibility? Does it make there lives easier?

Also are there any techniques or practices anyone can recommend, that help to convince the decider NOT to select the 'over-confident' person every time - I've seen people try to use data or numbers but it had no impact at all.