Isn’t part of being an expert recognising expertise?
Isn’t part of being an expert recognising expertise?
I'm thinking specifically here of the various stories that crop up when a grad student can't publish work that contradicts their advisor's own published work, but there are also plenty of examples in other fields as well.
At the end of the day, you have people who have either studied a thing or have direct experience working with it, and you assign them some amount of trust since they've done it more than you have. You don't need to know much about a particular sub field of biology to recognize that someone claims to have expertise in biology.
Knowing some yourself helps you see if they're making it up. Even if you don't know much, you can still pick up on whether they are open to new ideas, though.
All of this is to say that reducing "expertise" to a quantifiable binary answer is not an easy thing.
So there is already a certain selection bias in who would even become an expert and why.
edit: (I feel like Walter Lippmann went over all of this already in The Phantom Public, and came up with no good answers.)
But also, since it's relative to the population, the knowledge of "experts" does not need to have any bearing on truth. I thought this would be a problem in our context, but then I thought, that's not so far from reality anyway!
Expert identification can, to a degree, be bootstrapped out of a crowd using something like Prelec's Bayesian Truth Serum (https://www.science.org/doi/abs/10.1126/science.1102081). The tldr is that it fishes out holders of unusually surprising knowledge relative to the population. I was involved in a study that used BTS effectively to identify expert dermatologists.
I imagine it's possible to find a sweet spot between voluntary querying (a-la slashdot) by a rotating, small party combined with a variant of BTS which surfaces the experts as needed.
The cost of identification will still scale with the group, but it might be doable at ratios that are similar to moderator:users in forums.