Hacker News Reader

Comment by o_nate | Hacker News Reader

Good interview question.

That is a terrible interview question. First, it's likely to make candidates more nervous, rather than set them at ease. Second, it tells you very little if they can't solve it, or if they solve it immediately.

My rule is that you should stay away from aha! type questions that really only tell you one bit of information.

Designing a program to solve this kind of problem might be more of an interesting conversation…

How do you figure this to be an aha question? To me, It's a simple problem based on elimination of obviously incorrect answers. There is no "hidden information"/trick/leap of thought/insight that is required to solve it.

Logic questions are about as far from aha! as you can get. Next you'll be telling me that FizzBuzz is an aha! question. Sure, for some people it is, because they don't have a clue. Others are able to solve it systematically, and these are the candidates you're looking for.

The same holds for the logic puzzle in the OP. You can solve it step by step, without any leaps. The main difficulty is getting over the initial intellectual "flinch", and making yourself actually engage with the problem. IMO that's an important skill for developers to have.

I agree, although I probably wouldn't ask this in an interview just because of the risk of them seeing it before. At least when I was solving it it felt very much like debugging; you have a few snapshots of the variables and you have to reconstruct the whole logic flow.

I'm not sure that a good rule of interviewing is that you should only ask questions that put the candidate at ease. It's a job interview, not some soft-ball celebrity talk show interview. I don't think it's a bad idea to include a few tough questions. I agree it doesn't tell you as much if they solve it immediately. I think it might tell you something if they can't even get started on it.

Not really, no. It's part of a whole class of similar puzzles, and they're easy to solve formulaically, so all it proves is that the candidate has done these puzzles before (which I don't personally find to be a good indicator for job aptitude). It's also the sort of thing that will make nervous candidates who haven't seen such a problem choke without giving you any indication as to their quality.

But do you want your new employee to choke as soon as they see a new problem? To me, being able to solve the problem doesn't reveal anything; however, how they handle the problem can. I think its about context and what information you're trying to glean.

I recall reading that these sorts of problems were popular at one stage at places like Microsoft and Google for precisely that reason- they allegedly revealed problem solving abilities. They were subsequently dumped when it was found that they showed little correlation with job performance. I don't have the citations offhand, but I would probably personally be wary of working at s place that subjected me to brainteasers in an interview knowing that they were somewhat discredited already.

I'd like to see the data on this, or a variation.

First variation is with the candidates ability to solve the problem. Like you said, Microsoft and Google seem to have determined there's no correlation between that and job performance.

Second variation is with a panel of interviewers who propose the problem. Based on how the candidate responds, the panel chooses whether to 'hire' the candidate, based on their ability to reason about the problem and communicate, rather than on their ability to solve it.

I wonder if there's a correlation between the yes/no of the interviewers and the candidate's job performance.

By that logic you could have them do the problem under time constraints and constant interruptions, you wouldn't want a new employee that can't handle stress and distractions.

I realize you were trying for a reductio ad absudum, but actually I kinda like your idea, because it suggests a wide range of other promising ideas. Imagine a software company whose programmers were chosen for being immune to procrastination. They would run circles around competitors!

That also is why the GRE discontinued the old analytical puzzle section, which used to be 1/3 of the exam (along with math and verbal). The scores were bimodal: Those of us who'd done a lot of logic puzzles in school got perfect scores, and those who hadn't did much worse. It turned out not to be a good indicator of anything meaningful.

Yes, but mostly no.

I showed it to a few friends and a few of them had zero interest. Some tried for 30 seconds, then asked for the answer. Another wanted a hint. One said "I don't get it". One gave the answer right away and it was wrong. And some didn't get it after a decent step-by-step explanation, or even weren't interested in the explanation.

That says a lot, I think, about a certain mentality of a person towards particular types of problems, challenges and learning opportunities.

But I say mostly no, because there are hundreds of questions like these. Once you do 2-3 (and if you're in certain circles, you most likely have), the 4th or 5th become very easy, if they are even different (and not just a copy/paste) of ones you've already done.

In short, I think the question can be meaningful sometimes, give false positives a LOT of times, and probably give some false negatives, too. (e.g. nervous people who can't think outside the box in an interview boilerroom which is an a-typical work environment in my experience).

So yeah, I wouldn't personally ever use this in an interview.

I can't stand logic problems like these. They take no creativity to solve. They are senseless. They are boring. And as IkmoIkmo said, as soon as you learn to solve one, you can learn to solve any of them.

Only if the job is for a logic puzzle designer, otherwise this question is horrible.

I wonder if people see "Good interview question." and think that also was intended to mean "This is a good interview question that would work well consistently for a broad swath of candidates and should be given a high weight and/or be used exclusively to rule people out."

For those people, it might be beneficial to stop making such assumptions -- doing so seems to reflect a tendency to jump to conclusions.

There is lots of room for tough interview questions that only a percentage of people do well, provided that your interview as a whole does not skew too much towards any one question.

Personally -- and I don't need people to agree on this point -- I'd rather setup an interviewing process where people don't ace all the questions and that is considered ok. I'd rather see how far a candidate does across, say, 10 relatively deep, hard problems, even if they only do well in a few of them. In other words, I'd rather find out where a candidate excels rather than "average out" an interview so that "on average" candidates don't think it is too hard.

To sum up what I believe to be true: heterogeneity and diversity is good, both in your interviewing questions and your team. Don't be afraid to ask tough questions -- but if you do, think hard about how you evaluate the responses.

Two objections: Questions like this almost certainly do not reflect the work involved with the job you're interviewing for. Not many jobs involve sitting around solving abstract, standalone logic puzzles. Programming, for example, involves logical deduction, but in concert with a variety of other skills, and applied to software, not prose, as part of a complex development process. Maybe these type of hard questions exercise some of the same skills, but why not set tasks that involve exactly the same skills, e.g. programming assignments or some other problem taken directly from the role you're interviewing for.

Second, the circumstances in which you're asking candidates to solve these problems almost certainly do not reflect those in which they will be required to work should they get the job. Nervous anxiety, unfamiliarity, strong time pressure and intense outside observation are not conductive to good performance, and are (hopefully) not part of your everyday working environment, but all are present in an interview. So when you present difficult problems, you're not just selecting for people who can solve them, you're selecting for people who can solve them in conditions that will never otherwise arise.

I was able to solve the logic question in the linked article after about five minutes of head-scratching. But I am 99.9% certain that, in interview conditions, I would never have figured it out. I would have spent those five minutes of head-scratching fighting off a rising tide of panic. And the thing is, having had interviews and given interviews, I am not even particularly nervous during them, relatively speaking.

I disagree, I got it very quickly by writing the dates in order, crossing off all that were impossible after the first statement, then with those were left crossing off the ones that were impossible after the second, then there were three, with only one possible according to the third. Now what if I did not have a pad of paper handy when interviewing? All this question would do is predict if someone would enjoy a game such as Professor Layton more accurately than flipping a coin.

Reply on news.ycombinator.com