Hiring: The Lake Wobegon Strategy (2006) | Hacker News Reader

Hiring: The Lake Wobegon Strategy (2006)

googleresearch.blogspot.com

50 pointsawa29 comments

This assumes from the outset that you know the difference between a good future employee and a bad one. Restating the problem so that the hard issues are out of scope makes any strategy look good.

How to find the perfect car: Just pick one better than all the previous cars you've owned.

How to find the perfect date: Only ask women more attractive than all your previous girlfriends

All of those problems require one very important thing: an assessment of the prospective employee, car, or woman. I'm sure Dr. Norvig has plenty to say about evaluating prospective employees, but it is still the hardest part about hiring better employees (my opinion, but with no actual hiring experience).

Do you double your number of cars or romantic partners every year? Then the analogy might hold and the strategy might work.

"This assumes from the outset that you know the difference between a good future employee and a bad one." -- Not really; his simulation assumed the hiring process had error bars of 15%.

It should probably be a lot higher than that.

And that only further proves the importance of Norvig's point.

While interesting, I wanted to point out that this is very old (2006) and may have changed somewhat since.

Now that they've grown to a certain size and have offices in different places, I would imagine they (just logistically) have to hire at the "site" level rather than at the company level now, which would encourage each site to bring in employees at a different mean than the entire company itself (i.e. like the hiring manager issue).

Not really. I was given a choice of offices during my interview process in 2008, when they had more employees then than they do now.

I don't care what weaknesses you can find in this post or Google's strategy, the fact that they're thinking about talent in this way is impressive.

Typically HR and statistical analysis don't mix.

I worked with someone who told me about a hiring strategy he encountered at a former employer. HR came across some study that showed that people with high IQs were the most likely to have personnel issues (or didn't work as well on a team, or something like that). So they decided to avoid these personnel problems by giving job candidates IQ tests and only hiring those with IQs that weren't above the company's mean IQ. I kid you not.

Was this the Police Force. They regularly reject people who are too smart, because they would get bored in the job.

No, I think it was in the health care industry, but I'm not sure.

I've heard this story, but I can't find any evidence for "regularly". (I'm also not sure what form such evidence would take.)

The highest profile case I can find is of Robert Jordan, of New London, CT, who sued for discrimination because his job application was rejected for scoring too high [sic] on the IQ test. (He lost.) (http://www.nytimes.com/1999/09/09/nyregion/metro-news-briefs...)

It appeals to my own personal prejudices, but it was also 10+ years ago.

From the article:

"I was eliminated on the basis of my intellectual makeup," he said. "It's the same as discrimination on the basis of gender or religion or race."

If he got what he was asking for, employers would not be able to reject job applicants that were not intelligent enough for the job.

Would it be possible for the source for this simulation to be released? To allow others to play with this. Namely, I'd like to see:

* Effect of Microsoft/Netflix policy of periodically laying off the bottom n% of the company

* Effect of churn (which tends to hit both the bottom and top more so than the average).

* Seeding by initial founding engineers. Founders generally have to be strongly competent engineers (95th-98th percentile), but not very many are actually 99.9th percentile "10x" engineers (e.g., Brad Fitzpatrick of Livejournal, Bill Joy of Sun, Max Levchin of Paypal).

* Speaking of "Nx engineers", how about a logarithmic (with log_2) scale of skill rather than a linear?

Of course, there is also the question: an interview shows tests aptitude (and an error rate of the interview process implies error rate in determining aptitude), but more factors play a role in actual measured performance. I wonder if this is factored in to the metrics:

  foreach interview_score in range(1,5):
    foreach year in range(1,4):
      foreach performance_score in range(1,5):
        p ( performance_score | interview_score ) at year

A company very likely must (by law) have performance_score data and it should be easy to reconstruct interview score data (unless interviewers are sloppy about writing down their feedback and decisions are made by a hiring manager based on oral input).

Here's a more detailed simulation that I just wrote in javascript:

- I assume an error margin in our ability to evaluate employees.

- I assume an applicant pool that isn't necessarily centered on z=0.

- I evaluate z-scores, not percentile.

- I calculate the average number of interviews for a successful hire.

http://fizx.github.com/hiring/

I can't take seriously the assumption that skill level in this context is a scalar.

Yeah, I'd really like to know how they get the Y-axis. All of the google interview recaps I've ever read have made it seem as though candidates aren't likely to get the same interview.

That said, it could be that their interview questions are like the questions on the SAT, or GRE, with the difficulty being normalized in some way.

Hmm, when I looked up "The Lake Wobegon effect" on Wikipedia, it linked to "Illusory superiority." I'm guessing that's not quite what Google is going for here.

I was wondering about that as well.

In Lake Wobegon, "all the women are strong, all the men are good looking, and all the children are above average." The Lake Wobegon Effect is one where most people think they are above average at something they are good at.

In this case, I guess they're just hiring so that everyone is above average. It makes some sense, but it does have a darker double-meaning.

What I found interesting was that the no hiring manager strategy doesn't help much... also I wonder where they got the data from for other hiring strategies that they don't use.

That was just a Monte Carlo simulation.

If you change the parameters of the simulation, you will get different outcomes.

Here is how to get hired at Google:

1. Start and sell at least one company. 2. Set a world record - the more obscure the better. 3. Have a Ph.D - it doesn't matter what it is in. 4. Know at least one manager at Google. 5. Do all of the above before your 25.

Sure. Do that, and you can get hired anywhere.

1, 2, 3 are by themselves exceptional-- pretty much by definition, setting a world record in anything is unusual. Having started and sold a company means you've built something worth real money. (The Ph. D. is similar, though it has a different metric.)

[Really, it'd be a lot easier to study algorithms and graph theory and ace the interview.]

All this really is is an application of micro economics to your hiring policy. In an ideal world this strategy works but as others have pointed out, without perfect data, the noise builds up so that it is hard if not impossible to maintain a rising average.

1. Who says growth must come from hiring better or even more employees?

2. Does Google feel they already make the most of the talent they have?

3. Do existing Google employees feel they themselves are in the best position to shine?

Regarding #2: I think they're not making the most of the talent they have. Google does some great/cool things. But I expect a lot more of that based on the number of employees they have, and the average level of talent/IQ they supposedly have. I suspect the discrepancy is caused by the dampening effect of having a large infrastructure and bureaucracy, and a possible clone/conformity effect from their hiring process. I believe their hiring process mostly filters for the wrong thing. At least with respect to folks who aren't already famous to begin with.

For a lot of employees, the project they'll be working on and what their manager is like are major factors in whether they want a job. Wouldn't a "no hiring manager" strategy cause problems because of this?

I understand (at least in the past) Google makes it easy to move between groups, so you are really joining a company, rather than a group. In most companies if you don't like your manager, your best option is to find another job.

Reply on news.ycombinator.com