Hacker News Reader

Comment by kjksf | Hacker News Reader

He probably meant that people who do programming competitions are more likely to pass Google interview (they'll be better at doing algorithmic questions quickly) but are not necessarily better suited to do the actual job.

Just like hockey players born earlier in the year were more likely to be drafted.

View on HN

To expand on this:

* Prior to officially selecting candidates based on performance in problems derived from programming competitions, candidates who excelled at programming competitions were likely to do well on the job.

* That correlation was observed on a wide scale by employers, so many companies -- Google chief among them -- started incorporating such questions into the official interviews.

* Candidates now observed the change in employer interviewing methods on a wide scale and adapted their preparation methods. This fundamentally changed the pool of people good at programming competition problems in such a way as to reduce the correlation between the original signal (good at algorithmic problems) and the goal (good at the job).

* Overall, widespread acknowledgment -- and all consequent changes in behavior -- of the original correlation between the signal and the goal significantly reduced the quality of the correlation.

I think it's simpler than this. I don't think there is much outright gaming of the signal.

Programming interviews and programming competitions are very similar, much more similar than programming interviews and real world software engineering. When you are selecting top programming competition competitors, you are implicitly selecting people who will absolutely smash your (non-design) programming interview questions. This has little to do with their effectiveness as software engineers.

In other words, what gets measured gets managed.

So if you measure hiring candidates by their performance in programming competitions, everyone will manage their own skills towards doing better in competitions.

Yes but not just that. I think the bigger trend is that training oneself to excel at those types of problems meant (in addition to other things) one thing about you back then (that you were really into programming). Now that same behavior likely means that you want to get a nice job at one of the big tech companies, as a result of them publically selecting for that. These are fuzzy indicators to begin with, but they're definitely different fuzz.

> I think the bigger trend is that training oneself to excel at those types of problems meant (in addition to other things) one thing about you back then (that you were really into programming). Now that same behavior likely means that you want to get a nice job at one of the big tech companies, as a result of them publically selecting for that.

Thanks for the explanation! The idea being discussed didn't quite click for me until I read this

Google's data shows that their interview process produces a low number of false positives, at the risk of producing a high number of false negatives.

That is, despite passing on otherwise talented people, those people who successfully pass Google interviews go on to be successful at Google.

edit to add:

Couldn't find that original article, this article goes on to speak about success predictors: https://www.wired.com/2015/04/hire-like-google/

Was it a proper double-blind study substituting interview result for rnd() to make a hiring decision? And established correlation a few years down the line?

Because what you've stated: 'data shows' ... "successfully pass go on to be successful" - sounds like cargo cult science or pseudoscience to me.

Do you really need to do the double-blind study, though? The median person cannot program at all, so using rnd() couldn't possibly have better than a 50% success rate (certainly lower; 50% is just a round number that I'm sure is larger than the number of programmers). Is Google doing worse than that? I kinda doubt it.

Yes. You can do it at each stage as well. And we are not talking about hiring a general employee but a CS or programmer. So employ rnd() after a resume screen. Google has already found that GPA and brain teasers had no positive effect. This is similar and it would be an interesting experiment.

Google generally isn't known for pseudoscience.

You've stated 'data shows'. So my question was - was it a proper double blind study?

Because it is definitely possible to do it properly. Substitute the results (or partial results) of the interview with rnd(), use it for hiring decision for a subset of candidates. Keep this information confidential. Establish if parts of your interview process don't perform better than randomness a few years down the line.

It's possible to do. Only I don't think this was done. And if it was not done, and the method was some 'data shows' with hand-waving - it would be under definition of pseudoscience.

You're calling most scientific journals psuedoscience -- even the hard sciences, like astrophysics. Nobody's doing controlled studies of supernovas, they're drawing inferences from observational data. I don't think your definition of science is sustainable.

Even Google has a PR department dumbing things down.

Definetely. I arrived at the final stage in a Google interview which I failed and I can confirm that out of the 5 interviews three were based on puzzles that I later discovered are found in books for coding competions. On two I did a good job to work out a solution myself but I completely got the third wrong. Who prepares for this kind of competitions has a huge advantage in these kind of interviews.

That's how I felt trying to find a job in the bay area. If I wanted to compete with the top talent then I needed to prepare for the interviews and practice those problems.

Your interpretation is too weak. It is not just "not necessarily better", it is the stronger "on average are probably worse".

The reason is that programming competitions give more of a boost to your odds of being selected than to how well you'll do on the job. So people who otherwise wouldn't have gotten in now will, and will not perform as well as the people that they displaced.

Which is what happened with hockey players. Being born at the right time of year put you in a bucket with people who were slightly younger than you. Which improved your performance on the tests, but didn't matter once you all grew up. So slightly worse people at the right time of year displaced slightly better at the wrong time of year, and the average came out that people who got through and were born in the latter half of the year were actually better.

The hypothesis is that the older kids are bigger stronger but the younger kids are more skilled. And apparently this holds up all the way to the NHL where the skill edge overcomes age.

Reply on news.ycombinator.com