As always, it's an imperfect proxy. Nothing more, nothing less. But nobody has found a more reliable proxy yet.
Companies have not found a process at scale where they can train their employees to systematically gauge candidates.
Small and startup companies where leadership is still involved in the interview process can gauge candidates based on their own technical intuition. So it's false to say NOBODY has a proxy for good software engineering skills. The founders can also look at the candidate's public code contributions.
The problem is that this method does not scale. If you trusted employees to hire based on feel and intuition, you open up the door for people to bring in incompetent friends.
The real problem here is incentives. Founders have a financial incentive to not hire bad candidates. Employees don't, and that's why a standardized process is needed.
Is there any proof that this method actually works better though? I've heard plenty of stories of people in smallish (30 FTE or so) startups working alongside some really disastrous hires.
The process wasn't anything special and the questions weren't particularly hard. It was very heavily standardized though.
No system is perfect. You'll get bad hires no matter what, and miss good hires. The question is only how many.
As a manager in a scale up, my incentive is to hire competent people into my team so we can meet our goals. Sure, it's not the same as the company being mine, but it's a good enough incentive. I think a standardized process is better because it reduces unconscious biases.
That results in biased hiring based on "I like this person, they think/talk like me".
> look at the candidate's public code contributions
Also biased hiring on candidates who have the time & inclination to do extra-curricular coding at the required level.
> Founders have a financial incentive to not hire bad candidates
All companies have financial incentives to not hire bad candidates, the cost a bad hire on productivity and cost to manage them out is high. In fact, the incentives are so strong that the entire hiring system is optimized to avoid bad hires at the cost of missing out on good hires.
That's why the system is so frustrating - only strong candidates with clear signals get hired.
It's OK to say "we didn't hire this person b/c they gave an example of how they reacted to criticism that doesn't reflect our corporate values".
It's not OK to say "we didn't hire this person b/c I had a bad feeling about them".
That's what "culture fit" is about, not behavioral questions.