Hacker News Reader

Comment by edw519 | Hacker News Reader

Imagine interviewing carpenters:

Method 1:

  How many ten penny nails in a pound?
  Who makes better screwdrivers, Craftsman or Snap-on? Why?
  What's your favorite toolbox? Why?
  How can you tell when wood has been treated?
  Why should you measure twice, cut once?
  Which end of the hammer do you hold?

Method 2:

  Show me something you built. Describe what you did.
  Cut this piece of wood according to these specs.

If you needed someone to build your house, which method would you choose?

View on HN

> If you needed someone to build your house, which method would you choose?

Both. The first method provides better protection against talented bullshitters, the second provides better insight into the candidate's work.

There is nothing wrong with examining the level of details somebody has at their fingertips. If they don't have the specific detail immediately handy, watching (and perhaps collaborating) with the person to find detail-type info gives you insight into whether they have a suitable user-interface when you need information from them.

>The first method provides better protection against talented bullshitters

Assuming that your skill level is above that of a talented bullshitter it shouldn't be that hard to drill deeper in to the previous projects with specifics. You don't necessarily need to even know the field, if you're a competent programmer you should at least get a feeling of how skilled/technical the other person is. And I'm willing to bet that that information will be a lot more reliable than the "what package is Foo in ?".

IMO using the first approach you concede that you aren't skilled enough to meaningfully evaluate the previous work/ skills, which is fine for contracting small work on your home, but you should consider third party evaluation/insurance service for anything big. In a interview scenario using this approach tells me I'm being interviewed by technically illiterate person that can't or won't use someone with more technical ability to do the evaluation, and both can't/won't options tell me that this company probably isn't for me.

> IMO using the first approach you concede that you aren't skilled enough to meaningfully evaluate the previous work/ skills

Only to those who confuse a test on details with ignorance of a field.

There is a wonderful scene in The Seven Samurai. The leader is testing the prospective candidates one-by-one as they walk into the "interview" room. There is a junior fighter hidden behind the doorway who attacks each prospective samurai as he walks through the door.

Most of the fighters dispatch the junior fighter without much ado, and forgive the "interviewer" understanding that he must test each one of them. There is one candidate who stands outside the door before walking in and says (paraphrased): Please, skip the attack that I know you have planned; It is a waste of energy. He does this in such a way that it is clear he will dispatch the junior fighter even before fighting.

Then there is one candidate who is unaware and gets whacked over the head.

The scene is a wonderful summary of interviewing candidates. (And a great movie too, fwiw.)

>Only to those who confuse a test on details with ignorance of a field.

There are insightful details that you only catch by working with something, eg. If I ask you what will the following python code print :

    def default():
      print("using default arguments [1, 3]")
      return [1, 3]

    def append_five(a=default()):
      a.append(5)
      return a

    print(append_five())
    print(append_five([1, 3]))
    print(append_five())
    print(append_five([9, 7]))

I expect you to know the "detail" of python handling default arguments, even tough it's non obvious anyone with decent amount of experience in python has probably been bitten by that behavior, you'd have to be a master bullshitter to guess that one. Similarly there are many such nuggets that could be used to determine if someone has actual experience/skill with the language/tools vs. reading the brochure before the interview and did his homework in college.

But this is not the same as asking what package contains what class, and similar petty details that you think are good questions because you looked them up in the documentation before the interview. I would expect many good programmers to miss that and bad ones get it right. Knowing the answer to the above question vs. knowing the name of the Java clusterfuck naming convention IMO gives a very different level of insight in to the experience of a programmer.

Eww, that's a nasty bug, though Python certainly isn't the first language to have it. IIRC, Dylan's "init-value" for class slots did much the same thing.

But really, this question only tells you whether a candidate is lying about Python experience on their résumé. If the candidate answers, "I have no idea. But I have 5 years of Ruby experience, and many years of Perl before that, and I can be competent in Python within a month or so," this question tells you nothing either way.

I've always been delighted to hire a really good programmer. I'm usually not too picky about which languages they're really good at.

Well technically it's not a bug - it's by design :)

But I agree completely, which is why I said you should be able to evaluate the previous work trough the interview, even if you don't know the specific technology (eg. If I was interviewing someone for a Django/Python gig and he has RoR experience ~10-15 minutes of interviewing about a previous project that he thinks is representative would be enough to figure out if he's winging it or he knows what he's talking about, it's cool if someone misses something but he should be able to reason about things on the fly, eg. in the above gotcha I expect you to figure out what's happening when I tell you the result if you didn't get it right away).

I'm just saying - some details can be useful, eg. if I knew Ruby or Rails I could ask about a few similar details that separate the men from the boys - but asking what namespace is foo located in isn't going to provide useful information and puts you in the PHB class of people who don't realize that their lack of skill is preventing them from creating good evaluation tools and can't delegate the responsibility to someone skilled - the kind of environment I would want avoid at all costs.

That's ridiculous. There are so many things that determine whether a candidate is good outside of how deeply they know language X.

There is only so much esoteric knowledge one person can know and you can't expect candidate's specializations to overlap yours. You are being lazy by looking for someone exactly like you (because you obviously know you're experienced) instead of trying to figure out if the candidate is knowledgable in their own right.

No, if someone claims he has experience with Python, and I've seen a few of the self proclaimed python gurus, that question will provide a quick filter. And you don't have to get it right, but when I show you the result I expect you to be able to backtrack and reason about what's happening on the spot. And there are other advanced things, for eg. meta-classes, multiple inheritance and dispatch order, etc. Not things that you need to know on the top of your head but should be able to reason about when provided the context. And this really isn't that esoteric, it's the sort of thing that separates newbies from experienced programmers, if you've written a decent amount of python code chances are you will know at least a few of the edge cases. And that question is only in response to jpdoctor saying that details can matter, I was saying I agree that some details can help but others are just useless trivia questions that make you look bad.

Also the language was just the first thing that came to mind, you can use a common library/framework/tool quirk, etc. Anything that shows familiarity with the required tools.

If I had to deal with the fact that 85 out of 100 applicants got the "end of the hammer" question wrong, you can bet I'd always start with method 1 just to filter out the imposters. That would free up time to correctly use the method 2 with the remaining 15 applicants.

The "end of the hammer" question was a poor choice. Also, you could get that by asking them to pound a nail into a 2x4.

A better example of nano-questions would be "What is the A312X131/22 3-1/2 inch nail used for?" or "How do the Black & Decker DS321 and Rockwell RK7866 belt sanders compare? When would you use one over the other?"

Well, then we're in agreement that "end of the hammer" questions are reasonable just to weed out the obvious imposters, right? So the question is whether these "nano" questions are reasonable equivalents?

And I think "how do you inherit a class" is the "end of the hammer" equivalent for a language if you claim to have been using it continuously for the past two years. I'm much less happy about the "what package" questions, but if they're meant to simply get you to name a few major packages I think they're fine. You might get the package question wrong, but I expect you to get it wrong intelligently ("what package" questions were reasonable 12 years ago when programmers dealt with them daily, but eclipse does all that for you now).

I think of these as "what color is the door" questions. If you tell me you've been working at Company X, I expect you to be able to tell me the color of the front door, or where the bathroom is, or something about the place. Not that I care, it's completely irrelevant to your skills, I just want to find out if you're really the person you claim to be before I waste time with a serious interview.

Because my time is precious, and I want to spend it with the serious candidates.

I think in this case, "take this array of numbers and sort them" would do just fine to see if they know the basics of the language, much like "take this hammer and nail that nail" would for basic hammer knowledge.

Another option would be to email the candidate a chunk of average code while you're on the phone with them, then say "walk me through the code and tell me what it's doing."

"I think of these as "what color is the door" questions. If you tell me you've been working at Company X, I expect you to be able to tell me the color of the front door, or where the bathroom is, or something about the place. Not that I care, it's completely irrelevant to your skills, I just want to find out if you're really the person you claim to be before I waste time with a serious interview."

That's what references are for. FWIW, I can't remember the color of the doors to ANY place I've worked, except for our current office which has quite a peculiar door. And besides, how are you going to know that I'm not bullshitting when I say the door was green and was right beside the elevator?

The problem with nano questions is that they assume that these few tiny tidbits of trivial and often overlooked factoids are representative of knowledge on the whole, and thus are prone to false negatives. They are heavily biased towards what YOU tend to notice, not necessarily what others tend to notice.

You contact your interviewees' references before the serious interview?

It's cheaper to contact a reference than it is to bring someone in for 5 hours.

My references are people I like and respect, and I'm not going to impose on them any more than absolutely necessary. I don't even give out their numbers until I have a firm expectation of receiving an acceptable offer. (I make an exception for HR staff, because confirming employment and such is part of their day job rather than a personal favor.)

Besides, if you have the slightest clue, your references will all speak favorably of you. It's a test with very poor specificity.

Actually know a guy who just had to completely redo his deck because the contractor used the wrong nails on treated wood. So in that regard technicalities do have some signifigance.

Method 2 would have covered it with :

Show me something you built. Describe what you did.

When the person describes what they did that opens up questions around materials, methods, and technicalities. Just having them tell you what a certain kind of nail is would have been useless without also demonstrating they used it properly when needed.

The fact he was using nails rather than screws for a deck should have been the first clue.

But neither method would have overcome your friend's lack of expertise.

Imagine if 90% of candidates were actually welders who acquainted themselves with carpentry by flipping through a book on butterfly joints. It is far more efficient to ask a "nano" question to get done with an interview faster than to laboriously struggle through them trying to present a fine welding job as a woodwork.

Differences between programming languages are much bigger and more important than differences between different brands of toolboxes or hammers.

If you realise you made a poor choice of screwdriver it is trivial to replace it with a different one and carry on working almost immediately since it's operation is basically the same.

Choosing your language or platform poorly and running into issues later once you have an app close to release or already running in production is going to cause much more difficult problems.

Essentially our tools end up embedded inside our product itself.

It's close to impossible to reliably compare programming to any other profession simply because there is nothing else quite like it.

You obviously have never seen the work of a truly talented carpenter the work the wood shop turned out at my first job was stunning.

Some of the models (for hydrodynamic research) they produced in teak and other hardwoods where works of art.

Comparing screwdrivers and languages is inapt. A better example would be a choice to use a different building material than wood.

I don't think this is borne out by reality.

Making a general contracting mistake such as accidentally compromising the structural integrity of a building by making a poor choice on where to run the plumbing can easily result in a lot of work having to be re-done, at significant cost.

A friend of mine just became a registered carpenter which involves hours of study and exams to learn the technical specs. He's now allowed to work on more demanding and interesting projects and gets a pay rise, because his houses won't fall down because of using the wrong materials. My grandfather can build a hell of a woodshed but I wouldn't trust him to put together a 3-story townhouse.

Depends who I am. If I didn't know anything about carpentry, I would choose method 1. I don't have the specialized knowledge to judge his work.

Insert obvious parallel about hiring managers who can and can't program.

Neither.

The proper method is:

Here are the plans and specifications. Bids close in two weeks. Please include qualifications and references with your submittal.

The first round of questions you would use to establish whether or not the candidate will have the vocabulary to work with the other members of your team. There is nothing worse than hiring a carpenter who can't communicate with the other carpenters he's building a house with.

Once you've established you can talk to this guy (and he can talk to you) you'd ask him to show of his skills.

IME the interviews have been more like "build this Escher painting with balsa... You have 2 hours".

I think the comparison of programmers to carpenters is really pertinent. There are so many analogies.. the modern computer programmer, as-is, is today's carpenter.

Reply on news.ycombinator.com