Unskilled and Unaware of It (1999) [pdf]

Yes - by reinforcing in people that confidence is disjoint from competence, and that 'trust-but-verify' is the best policy when evaluating people's competence.

So when someone says they are good at something, don't just take their word for it.

While I think there are in fact useful insights that the individual can glean directly from this paper, we shouldn't forget that scientific papers exist for a larger purpose than to be the tools of self-improvement or the weapons of our social wars.

That idea seems positive to me. If we could change our feedback mechanisms (via education and so forth), we could increase self-awareness and reduce incompetence.

Also, we tend to think of "incompetence" as extremely negative, but they use it in a specific way that is meant to be descriptive and not an insult.

That's my take on it anyway. I really don't know.

It is useful in pointing out why I prefer medical professionals who express doubt vs those who are super confident.

Case in point: Physical therapists and massage therapists. The farther you go from what your hospital provides, the greater the certainty and confidence signaling. Same with a lot of alternative medical practices. It is effective in explaining why so many people flock to such professionals - because they always diagnose you and tell you what your problem is. And then the patients turn around and say "These doctors are incompetent, and their methodology is a sham. They could not tell me what my problem was, with all their years of training, whereas the <insert alternative professional> figured it out in just one sitting. And it all made sense, too!"

1. It keeps me aware, especially in domains where I'm a beginner, of how it can affect my self-evaluation of performance.

2. It helps me when teaching beginners; I may need to help them realize where they lay in performance; if you don't know you suck, chances are you won't improve.

3. I've 'counseled' many people who are in the upper-middle of performance, when they don't realize how good they are :) this has helped them go for more challenging jobs etc

I like the paper, because it's a good reminder to seek out tutors / mentors early when learning something, and to take them seriously when they say you have a lot to learn.

I suspect that's exactly why it's popular, and why the critiques and followup papers that show this result doesn't hold up to real-world scrutiny don't seem to have any effect on the public perception of this paper.

It gives people an excuse to feel superior. I think it's a meta effect, because it's one of those specious, great sounding results that is super easy to understand and super easy to believe, but isn't actually true. Everyone has someone in their life that is aggressive and annoyingly over-confident.

This paper has some really big flaws, it's probably a bad idea to apply it to any real world situation including opponents. It can't benefit society to have everyone making incorrect assumptions that confidence proves a lack of skill.

At a minimum (assuming that it held up; it's been widely criticized since) it would tell us that self-assessment, even aside from intentional dishonesty, is a dangerous basis for assessing ability in intellectual domains.

Conversely, it also would inform us that people that mis-assessing their own abilities in certain ways may well be honest rather than padding or demonstrating false humility.

but then as you advance, as you acquire more skills, you also improve your ability to gauge your own competence.

so, clearly your only option is to not progress in life at all, or you risk ending up with your dreaded impostor syndrome /s

After all if you start projects with a YES attitude you are more likely to complete the projects, than if you don't start any projects because of your NO attitude.

Sometimes == not always; being competent is generally good thing :)

https://en.wikipedia.org/wiki/Dr._Fox_effect

Humans cannot evaluate competence objectively without formal performance testing. (And sometimes not even then.)

But we can evaluate confidence, charm, persuasiveness, and social proof.

So we tend to use them as a proxy, and choose leaders who lack competence but can demonstrate them, over leaders with genuine competence who can't.

This is a very bad thing, and possibly the single biggest cause of political, social, and economic problems in our history.

That being said, there might be some statistical magic hidden in the graph, where the group, the further up it goes, does less rely on self-rewarding by guessing a lower score and splitting up into "actual" self-underselling for mimicking, evening out with the ambitious but really good ones.

What's supposed to happen, then, is that any interesting findings will then be replicated using a more expensive, higher-quality study. That has kindasorta happened - there has been quite a bit of follow-up research, but it tends to still be performed on undergraduates.

All that said, even if the population they studied is only undergraduates, the findings they report in the paper would still be interesting even if they were only useful for improving college-level educational programs.

http://www.slate.com/articles/health_and_science/science/201...

See below for the ref and a link to the article.

You, I, Dunning and Kruger all understand the limitations of experiments with local undergrads and why it's done. I respectfully suggest that D&K might understand that better than you or me, given our skill sets.

>It's possible; I can't ignore the possibility.

Yes you can ignore it, just focus on things that actually matter. While other question their competence, you can actually get stuff done.

The most skilled people were only off by like 10%. The effect is just that people seem to think the curve goes from 0.5 to 0.85 instead of 0.0 to 1.0 but otherwise fit themselves in the same rank-ordered buckets within that.

This paper is only valid for what it actually studied: some subjective tasks and some standardized test questions. Ability to get the humor in a joke was one of the 4 tasks. Subsequent papers have shown that this result doesn't apply to highly skilled or complex cognitive tasks like computer programming. In fact, the effect reverses, and highly skilled people are disproportionately better at knowing their competence. https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...

This paper also didn't control for a possibly over-confident sample: the subject of the study were all Cornell undergrads, who may be prone to being overconfident, or just acting overconfident.

That said, as a programmer, it's always a good idea to be confident about your willingness and attitude while being humble about your skills. Assume your programs have bugs, because they do. I believe the reason my competence is increasing is because I assume all code is wrong, and I take extra time to make sure it does what I expect.

Anyone know of replications with a wider pool?