This is a good point. I generally like the “show us the code” method of social proof in open source communities much more than the “show us your certifications” method in academia. The problem of course is that the latter is much easier to scale up, because the certification stands in for a close and careful examination of each person... “Oh, he graduated from X school? he’s probably alright” is much faster than “Let me spend a few hours reading a major piece of work.”
It’s interesting to wonder whether how you’d go about building communities for doing science in a way that had tasks for inexpert but hard-working/bright newcomers to get started on, and allowed building reputation without all of the hassles of degrees and academic politics where much of the effort is misdirected. I wonder when/if some of the methods and norms of online communities will begin to supplant those of academia as it is today.
Interestingly, when it was smaller, science used to be much more accessible in this way. Once you were literate, you’d passed enough of a bar for serious people to treat you with some respect, and all kinds of advances were made by amateurs. So are the more open systems in computing just because of the youth of the field, or does online communication allow amateurism to scale further than it used to?