Why we don't hire programmers based on puzzles and tricks

My gripe these days is that places I've interviewed with now that I have a few years of experience are still concerned about big-O notation and a lot of the more academic questions. I feel like I shouldn't have to go back to my college notes to prep for an interview for a job that I'd really be using my current experience for. I think it comes down to other developers not spending time really thinking about the quality of their interview questions. They just do a search for 'programming interview questions' and call it a day.

My most recent interview was with Amazon. I was asked the dumb big-O runtime stuff as well as how I'd count all of the stars in the universe. It pissed me off, but I did like the question I was asked about how I'd design an elevator system and then asked how my solution would scale. That's not too far off from what I'd be doing, albeit in an abstract form.

The interview process is so badly gamed that these days its kind of a ceremony. In a recent interview, I gave a candidate a practical problem. He was allowed to use the sed man page, and then I gave him a file and search/replace problem. Nothing much! The candidate fails! Instead he asks me if there is going to be a algorithm/data structure interview. I told him, I don't know but as far as I was concerned I was only going to test him on practical everyday problems.

It was almost like I had committed a sin. He was like, every company has those standard one's so why was I doing like a practical test. These days all candidates do is go through most common puzzles/algorithms/data structure stuff. Its actually quite easy to get access to those. Just spend some time on interview forums and you can practically game any interview, in any company any where around the world today.

My approach is straight simple. Throw a manual/documentation at the person. Give him a problem common enough to test a practical everyday situation. Check if he can pass the the test of reading the documentation and figuring out the solution to the problem. Or give him/her their favorite IDE/text editor + documentation + problem and let them solve it.

If they can do it, or even get the approach right they pass the test.

Else if all they are going to do is vomit some mathematical theory from college text books, which isn't even remotely relevant to our everyday work- I'm hardly interested in hiring such candidates.

I can sort of see how implementing my own hash table fits into a dev job, but I why manhole covers are round, or light bulbs in a room has absolutely nothing to do with anything relevant in our field.

People who use logic puzzles as proxies for anything in an interview have my everlasting scorn.

Asking a candidate a difficult brainteaser that may be even close to impossible can be extremely revealing. There isn't a canned response, you get to see them work under pressure, you experience live problem solving and you can understand how they move through their work better. All of these factors are just as important as competency when hiring.

A great programmer doesn't "fizzle" in front of a difficult interview question. He/she may or may not get the correct answer, but as long as they have the requisite problem solving skills and communicate with their interviewer well they will come out fine. I'd say that showing how you fail can be quite beneficial when trying to get a job, especially if you happen to fail gracefully :)

It seems like 37signals is just echoing platitudes to gain karma/publicity at the expense of cheapening the discussion.

My first job wasn't a programming job, but I did a lot of programming because the need arose and I was the only one capable of doing it.

I always felt too inexperienced to look for a real programming job. A friend pushed me to interview for the position I'm in right now (full time web developer). They didn't make me do any whiteboarding or puzzles. What I did do is give them a code sample and we discussed how what I could have done better and why.

I got the job and I'm doing quite well, despite my lack of a CS background.

Since I don't have a formal CS background, and most of my development experience is web development, I have basically no experience dealing with complex algorithms, but I've found I can solve a hard problem with enough effort (Google, co-workers, etc).

For web developers, wouldn't more appropriate questions relate to "please design a database for a website that needs to track the following", or "What would your process be for designing and implementing a REST API that would...". These would allow the developer to show you their thought process of working through a large problem without there being a necessarily right or wrong answer.

Also I don't try to be confrontational, I take the "let's solve this together". That often puts them at ease and makes for a better experience and a more productive interview.

Now it is still an interview and the selection process could be brutal but this way I get the most _relevant_ knowledge and information about the applicant in the shortest amount of time.

Let's just summarize the points:

- Not everyone can produce real-world code. Most of what we do for a living belongs to our employer. Side projects, particularly in the US, can be an issue for IP reasons (California is somewhat of an exception here);

- Side projects, even open source projects, can be of questionable "real world" value;

- I've personally encountered many "programmers" who can talk a good game but can't code a for loop;

- tests like the FizzBuzz test [1] are, in my experience, remarkably effective as an early negative filter. This is an important point. If someone blows you away at FizzBuzz, it doesn't mean they're an awesome engineer. But if they can't do it, it almost certainly means they aren't. The idea here is to spend the most time with candidates who might potentially work out and wasting as little time as possible on those that probably won't;

- the problem with these kinds of whiteboard coding problems is that the tendency is for interviewers to think the problem needs to be "hard". It doesn't. In making it too hard (IMHO) you risk destroying the value of your filter;

- pop-quizzes of obscure language features, the kind that might appear in certification exams, are a waste of time. I have no argument with that;

- whiteboarding code by itself is not a great filter. It should be used in conjunction with a multi-faceted interviewing approach that involves testing fundamentals, the ability to construct a relatively simple algorithm, the issues of working on a team and on a production code base and systems design.

- the problem with simply talking about "real world" code, as the author suggests, is you're no longer finding a good engineer, you're finding someone you like, someone who thinks like you. This falls under the umbrella of cultural fit, which is of course important, but don't mistake that for engineering skill.

- I think we can all agree that "logic" puzzles like "how would you move Mount Fuji?" or "if you shrunk to 1cm in size and dropped in a blender, what would you do?" are stupid.

- testing "back of the envelope" estimation however can be useful. I mean things like "how much storage is required to store satellite images for Google Maps?" The idea isn't to get an accurate answer. It's to see what assumptions the candidate states and, based ont hose assumptions, to come up with a reasonable ballpark number.

The problem here is that there are many engineers who can't comprehend the possibility that there is someone being paid to be a programmer who can't code. But I assure you this is the case. It's shocking but true. Simple coding tests largely filter these people out so if you're offended by such simple tests, just do it and move on. I assure you there's a reason why they exist.

One final prediction: There's some guy here on HN who always posts the exact same huge comment on any hiring thread. I'm sure it'll pop up any moment now.

EDIT: let me add a point about trial periods and take home assignments.

Both of these are guaranteed recipes for mediocrity. Truly outstanding candidates need to justify the time investment for either option and very few companies have the kind of gravitas that would justify it.

Anything written without supervision will be of questionable provenance at best.

As for whether or not someone will work out in your organization, bringing them in for a day is (IMHO) of questionable value. Many engineers are introverts. I include myself in this. It's incredibly awkward as is to be in a new company or even a new team in the same company. I question the value of any such assessment over what you learn in 1-4 hours of interviewing.

[1]: http://www.codinghorror.com/blog/2007/02/why-cant-programmer...

I don't understand the 37signals love sometimes.

"Given that when you're duck hunting the key is to shoot ahead of where the duck is, and that a lot of the suggestions have been the next iteration applied to the hobby/pastime... what's the third or fourth iteration?".

Basically I want to see the excitement and passion about technology, their pastime, and how they think, how they identify problems, solve them in numerous ways, go beyond just today's solution. And I get wrapped up in it too, I also get excited by some of the stuff that comes up.

With a great candidate... the interview ends with us both inspired and a thousand new thoughts floating around the room.

The vast majority of candidates cannot leap beyond the first iteration, and some struggle to even see how anything within their hobby could ever change.

It was a codility.com test - I had not actually been expecting it but, kerpow. I rewrote my own abs() because I forgot such a thing existed, the problem itself is still blanked from my brain - and every passing minute it got worse.

After an hour and a half, the interviewer took pity on me and drove me to the station.

I actually sat on the train took the codility demo test. 100%, 3 minutes. Signed up, took more. I could do them. Just impossible to know what went wrong but it went badly wrong.

Ultimately it worked out well. I would have had to live weekdays down there and my marriage probably would not have survived it. Now I work 15 minutes walk away and give my kids breakfast each day after we sit on the sofa and watch Mister Maker.

For me that test was a wake up call.

Firstly, I run my own business - being at the mercy of one boss is rubbish.

Secondly, these tests are rubbish for deciding who to hire - but they are OK for programmers as a form of continuing education.

Thirdly, the exercises at the end of each chapter of SICP are much much better form of education.

Forthly, I was once asked how a large corporate IT shop should handle a reduction in workforce. I said march everyone into a room and those who cant code FizzBuzz get a pink slip

Even given my experiences above - I still think that is the best option. Sometimes a guy just dies on that day. Its unlucky. But sometimes it works out for the best.

edit: clarity

After the interview I email them a small unfinished program. I then give them a set of instructions on what I would like them to do to finish it off, and they can then do it at home on their own dev setup; with the internet and any other resource they would normally use for development.

I give a coding style guide, and in the areas where some specific technique is required I make that explicit.

The app has a few bugs in it, and some of the techniques asked for are slightly leftfield.

I instruct them that if at any point they have any questions then feel free to email me - and they won't be marked down for asking questions.

In fact I mark candidates up if they ask questions.

The task shouldn't take more than 30-45 mins for a decent coder.

I find this to be a really valuable approach to seeing how a coder works. It shows how they work with others' code, it shows they can follow instruction, it shows they can communicate when unsure rather than hiding away and it gives a good guide to their style of coding.

It's a far less confrontational approach, and gives the candidate the best chance of showing what they can do.

In the interview I tend to be much more interested in the person, because I'd rather have a coder who is slightly less capable that I can train, than a coder who is amazing at everything but won't fit in. I'll still discuss past projects and dig deeper where necessary, but I like to find out how they influenced the outcomes of projects rather than get too deep on algorithms and the like.

So far it's worked very well :)

Of course, you could always take smart people and train them - but seriously, who does that anymore?

I sketched out a quick ERD, and explained the high level architecture of the system. No code was written -- it was assumed that I could implement said system if I could design it.

The worst job interview grilled me on low level algorithms (I'm not a CS grad) and wanted me to sketch out a quick sort. Of course, I can describe a quick sort now as a result of said interview, but since it was all C# I doubt we were going to have to implement our own sort algorithms.

Another terrible interviewer asked me about projects I did in college. I finished grad school 3 years ago, and have 8 years professional experience...

We also found puzzles to be a useful talent magnet. If you're Google (or perhaps 37Signals) you don't need a talent magnet. If you're a small company nobody's heard of, one can be quite helpful.

I say because the best places I've worked at already have a "system" in place for their front-end people. You can still be a marginal developer and do well if you follow the system. It takes a few months, but once you know how they write their code and the standards they use, it's pretty easy to write sustainable, reusable code.

This also makes it considerable easier to think and work outside the box. You can take a few of the better programmers, tell them about a new framework or approach you want to use on a project and let them loose. If it's a success, you merge these new ideas into the existing ecosystem. Simply repeat as necessary.

The industry really puts a lot of hard work into finding and hiring good candidates, but it's not the programmers who are important. It's the system you're plugging them into which matters.

[1] Eric Lippert summed this up quite nicely:

>> Does not having knowledge in Data Structures really affect one's career in programming?

> Well it certainly will prevent you from getting a job on my team. But like I said before, programming is a huge field. There are lots of kinds of computer programming that don't require knowledge of data structures.

[1] http://programmers.stackexchange.com/a/102123

If you can't solve that, even in the pressure of an interview, then I'm sorry, but you have no business working as a programmer, and I'm offended that you even applied.

And yes, people flunk it all the time.

1. Junior developer code review where a save method included a call to a list method "because it's more efficient" than making a separate ajax call. Explain to the junior developer why that's not necessarily a good idea.

2. UI wants a tree view of categories where each category has a name and a parent category. The number of categories can be arbitrarily large. Discuss problems you might run into using a database agnositic approach (e.g. Hibernate) and what solutions you'd propose.

3. Prototyping a new app, two potential clients in the same domain have widely different ideas of how those domains should be represented (i.e. obviously similar domain class names, very different properties). Discuss ways of modeling domain classes such that each client sees the data in their preferred format.

All in all, I think the interview went pretty well because the discussion questions led to discussing all sorts of related topics that gave me a good feel for the types of problems the candidate had run into and if he had success in dealing with them.

It was great giving an interview without arcane things like explain the yield keyword in C# and how it might be used.

The point in the submitted blog post by 37 Signals, "The only reliable gauge I’ve found for future programmer success is looking at real code they’ve written, talking through bigger picture issues, and, if all that is swell, trying them out for size," basically says that you should do a work-sample test to hire a programmer. And that's what research says. The more a hiring manager understands what a worker will do on the job, and the better the manager appreciates what a new hire may grow into doing after a few years on the job, the better the manager can devise a work-sample test to identify a good worker. There is a research base for this. Work-sample tests aren't perfect hiring procedures either--nothing is foolproof, and every hiring procedure goes wrong with both "false positives" and "false negatives"--but you can improve your odds of hiring a good worker by using a work-sample test routinely whenever you are hiring. As a job applicant, you can select better rather than worse bosses and co-workers by aiming most of your applications at companies that explicitly use work-sample tests as part of the hiring process.

With the help of several Hacker News participants, I have written a FAQ about company hiring procedures, revised over a few months of edits to fit the typical recurrent HN discussion of this issue. See

http://news.ycombinator.com/item?id=5227923

for the latest posting of that, with full references to the research literature and legal cases about hiring workers in today's world. Feel free to contact me through my HN profile

http://news.ycombinator.com/user?id=tokenadult

if you have suggestions for improving that FAQ before I post it to my personal website.

P.S. Even before I saw the prediction at the end of Cletus's comment, I planned to make the prediction untrue.

EDIT TO RESPOND TO FIRST REPLY ABOUT PUZZLE QUESTIONS: Yes, if a company in the United States insists on using puzzle questions as a hiring procedure, and justifies using those puzzle questions by saying that they want to see which applicants are "good problem-solvers," or "able to think on their feet," a rejected job applicant just might be able to subject the company to a very expensive lawsuit based on employment discrimination, unless the company has prepared beforehand a validation study showing that those puzzle questions have a bona-fide relationship to work requirements. I would not advise a company to take that risk, especially when the legally safer alternative of doing a straight-up work-sample test is available. The law is different in other countries, and as a reply in this thread points out, in the EU it is generally legal to use straight-up IQ-type tests in hiring processes, although those are underused by private companies in Europe, according to the sources in my FAQ post.

ANOTHER EDIT, TO LINK TO AN UNBELIEVABLE ANECDOTE ABOUT HIRING:

In August 2012, I heard a story from a hiring manager of programmers about the hiring procedure he uses as an initial screen for applicants who have degrees in computer science: "Write a loop that displays the numbers 1 to 100." That sounds awfully easy, even to me, but he says that the great majority of his applicants with accredited CS degrees fail that screening test. My earlier telling of the full anecdote

http://news.ycombinator.com/item?id=4603414

and his

http://news.ycombinator.com/item?id=4919749

seem pretty nearly unbelievable in what they imply about how clueless many CS grads are, and yet I think the anecdote is a true description of reality. Cletus too mentions in this thread, with considerable agreement from reply comments, that many people hired as programmers cannot actually program.

Think of a problem you had before and solved or are attempting to solve. Ask the person being interviewed how they'd handle the problem. Do they have good process, intuition, and problem solving skills? Are they clear about communicating their intentions and explaining their decisions?

This isn't a trick - it's what the person is going to be doing there, in and out, every day.

Things like "write a function that performs merge sort", are bullshit because you would never do this in real life, largely because languages have sort methods that are already extremely performant. Plus a lot of languages will already be using some variant of merge sort and exposing it as a "sort" method.

If you are given a problem like "create a tabbed news component" or you're asked to do a small project (nothing longer than an hour), I think you can still get a good understanding of how people solve problems. This is less stressful, less "gotcha" mentality, and a fair approach.

So did I. (Well, it was C then, but same idea.) The prospect of scrawling code on an inapplicable medium while people stared at every move is, of course, irritating.

Instead of going to the whiteboard, I pulled out my notebook computer, fired up a code editor, and told the several guys present to continue the interview _while_ I worked on the challenge. Time was saved by multitasking, I did the work in a sensible normal manner, everyone was comfortable with the situation, and they were happy with the resulting code.

I got the job. And turned it down.

Testing with excessively clever tests and puzzles will attract a mindset of creating more complex code and than needed. Architecture a problem in a way that solve the problem elegantly but remains open and flexible is far more a tougher challenge than an algorithmic challenge.

There's a fine line between testing a hire to make sure someone's behind the steering wheel vs. communicating that you want things solved in interesting ways.

First off, as I've become a better programmer, I feel this fact is best shown in my opinions. When I first started out, if you had asked me my opinion of PHP vs Python, I would not have been able to say anything coherent. Now that I've worked with both technologies, my opinion is much more coherent.

Instead of asking puzzle questions, ask candidates their opinion. "What is your opinion of Mongodb?" I've spent enough time in the trenches with Mongo to have quite an opinion of that technology. Same with Postgres, Django, Javascript, Coffeescript, etc.

I think hands down, the worse type of interview questions are the ones where you're told to solve a problem, but you can only solve the problem in a certain way.

Its become a interview idiom for me. One recently was to reverse the words in a string. In comes "This is a string", out comes "string a is This". No problem. In python this can be done with one line:

the_string.split()[::-1]

It takes me 5 seconds to write that out. The interviewer telle me "good job", then he asks me to do it again, but this time to do it without using any standard library functions such as split or reverse.

At this point, I can almost with 100% certainty that this company will be a terrible place to work. It wouldn't piss me off it it wasn't for the fact that like 90% of all interviews I do end up like this.

I've through about this a lot, and what I've come up with for an explanation as to why I can;t do these types of questions is because my algorithm writing process is very subconscious. When I'm writing code and I come to a problem that requires a lot of thinking, I usually stop, do something else and let the problem float around for a bit in my subconscious. I got my best ideas when I'm in the shower, on my bike riding around time, even while reading a news article. When people are watching me (especially strange people I don't even know) I don't do my best thinking. At this point I'd do anything to be able to think in front of people. Because its keeping me from being able to get jobs.

I walked out of my first interviews, because 'I do not like programming games.' The first was a 4d matrix puzzle with more mathematical/theory roots than programming/logic.

My third interview started with the above quote and I was hired on the spot -- I believe they took it as a rebellion of every fresh out of college kid wanting to program video games.

I've done my fair share of interviews. I still don't like them. I've narrowed it down to basically two things:

1. There is a lot of highly subjective, mystical advice about what "works," in a technical interview and what doesn't. Big problems, small problems, trivia, puzzles, white-boards, no white-boards... hardly any of it is thoroughly studied and built on solid theory. It's all conjecture and personal bias.

2. How the candidate is feeling and where their head is at on the day of the interview matters. I can go on about cache alignments, bus errors, segmentation faults, C++'s lack of order in evaluating function arguments, why template class declarations must exist in the same compilation unit as the definition, why CLOS is amazing and the computation model of recursion is brilliant (or at least why I think so). Yet on a bad day I can (and have) botched perfectly trivial, benign problems.

However at the end of the day you need some sort of process. I just think that the current practices are not good enough. It seems to me that companies are probably turning away perfectly fine candidates without realizing it. There's pros and cons to every approach I suppose. But I still think trivial problems are dumb.

Don't forget it's not always how smart they are, it's their work ethic and whether or not they'll be a good fit for your team. Personalities that meld well with already hired employees are an essential must, no point hiring an introvert for a position when the team are extremely social people who have Friday afternoon beers.

Here is a tip: hiring a developer? Ask them to write a blog application, a task management system or a real world problem they will encounter if hired by your company not some ridiculous test that could stop you hiring a great developer. I consider myself to be pretty good, but I don't have a CS degree, I suck at mathematics and can only do simple math, yet I am able to produce exceptionally great code especially in instances where deadlines are tight and sometimes unreasonable.

So, as companies like Google and Facebook have so many applications, they're much more concerned about false positives than false negatives.

It's not about the actual code you produce in the interview, it's about talking through your thought process, your ability to reason while under pressure, and your attitude of whether or not "this is beneath me - wah!".

I've been in several interviews where I had to write code. I made some mistakes, some things I plain out didn't know (for my first job, I had no clue how to write any JavaScript, and I explained that it was just something I didn't know how to do, but that I'd be willing to learn it as required). I still got hired from all of them. So, if you think coding tests during interviews are about producing flawless code, they're not. They're to see how you think - and that is profoundly important as a developer.

I would also argue that a developer's existing skills are the least important factor when deciding whether or not to hire someone. Every developer works as part of a team and has to collaborate and communicate with others in that team. If you want to hire a developer, you need to see how well they communicate complex technical problems to non-technical people.

I probe for weakness in 5-6 different areas of computer science in the phone screen - keeps me from bringing in weak candidates.

Then in the on-site, I give them my Macbook and ask them to solve a few simple problems with some flat files I conjured in a language of their choice that's installed on that machine (C/C++/Java/Python/Ruby/PHP/Perl/shell/etc). You get an idea of how they solve problems, think about solutions, fix bugs, avoid common errors, and what they look up on the web (I let them use a browser to lookup apis and so forth - it's a good way to weed out "cut and paste" programmers). The people who do well finish quickly and we chat about the company with the remaining time - the people who do poorly use almost all the time on the first problem, make a bunch of mistakes, and usually don't understand the proper data structure for the job.

There are a few programmers that have a proclivity towards "clean, perfect" solutions, but FizzBuzz is ugly.

Making FizzBuzz work at all is just stupidly simple; but the moment you ask yourself about the little details, you notice that you either have to make an if tree, a bunch of if-elseif's, or some other hinky thing that ruins the "purity" of this simple little program.

Alot of people take the perspective that if you go

    if(i % 15)     print "fizzbuzz";
    else if(i % 5) print "fizz"; //or was it buzz?
    else if(i % 3) print "buzz"; //or was it fizz?
    else           print i;

I suspect when presenting a problem like this, the interviewer filters out two kinds of people: the people who "can't code", and the people who are obsessive.

The other part was being shown intentionally messy and broken code. Not code that'd fail to compile (although it sure looked like it would), but something thrown together in a completely insane manner that I couldn't possibly make sense of in 15 minutes. Much of it boiled down to, "What would happen if you call this function and use this variable before you actually define them and then do x, y and z?", and the way this language did it was quite different from other languages. I mean, yes, I did learn a lot about the language from that interview, but I'd really hope that's not the kind of code they'd be throwing at me or expecting me to write at their company.

By the same token, I've been interviewed myself where I've been given weird shit questions like others have mentioned; not the blender one, but shit along that line. That's nothing I personally do without some serious contemplation, not to be achieved in 5 minutes in an interview. Anyone who does so likely knew the answer ahead of time or is extremely puzzle-oriented. I think the real test is to examine the puzzle-solving processes of the candidate, but as a rule I don't think much of those kinds of questions.

Personally, I use a whiteboard when hiring. It's not the deciding factor... but it's a part of the bigger picture. As part of a small start up I need to know right then and there if they know what I need them to know.

Well this rarely happens , to start of folks (generally) tasked with the hiring process are not the right "vision" folks. They are highly task oriented individuals aka MS Project specialists.

A real test would have to encompass among many things. - Do they are ask the right questions ? - Are they capable of picking up new languages, ideas , technologies while defending or questioning changes in an articulate yet respectable demeanor. - how they handle things when they get to a wall.

So the question is what is the value of a skill or knowledge that fades away after a month or two? I know people who got hired by Google or other big companies that forgot all the information they acquired before the interview. Cause most of it is useless for the actual job. Many logic questions or puzzles are well known and people tend to know the answer before the interview.

If I had to interview someone; I would assign him a project from the company that is part the of the job he is going to do, leave him alone in a room with a computer for a time period I expect someone to finish that specific project. Finally, I come back to see the out come.

It is good for the applicant, as there is no pressure on him (except some time constraints maybe which is not a big deal), I do not put him on the spot and he does not have to deal with his boss while answering to a question. He does not have to dig unrelated information before the interview, and he can show off his programming skills if he has any.

It is good for the company, as I do not have to spend a day interviewing and prepare for that, I will know right away from the code if the guy is a good fit or not, I can see his programming skills, I know ahead of time how long in average it takes for an internal person to do the job, so I can measure his performance relatively. I do not even have to evaluate the code in front of him, I can invite multiple candidates at the same time, put them in separate rooms. Get the results, evaluate them and invite them for the second round if I liked their code.

I don't care for puzzles. Just conversations. I skim resumes looking for glimpses into personality more than looking for skill sets. I have a guy that went to Yale on my team. I didn't even know that until he mentioned it the other day, yet it was on his resume. I missed it because I don't care where you went. I don't care how fast you can solve a puzzle. It's put up or shut up and show me what you've built when you come here. If you can do that in an interview there's a great chance I'll hire you. The last two hires whipped out apps they'd built and then we went over how they accomplished it. To me that's more valuable than any other process, having a portfolio or sample ready when you show up.

"Of course we do not write code on whiteboard everyday. Of course we do not write code to detect if a graph is acyclic (my interview question) on a daily basis. Of course we do not put our engineers on the spot with these types of questions. However, when the time comes, may it be only once in 10 years, we know that we have hired the right people because these questions show your thinking abilities - not if you know the answer or not."

Btw, I didn't get an offer, but I'm going to Amazon where I had very similar interview process as Microsoft.

I want the right personality. The ultimate skills will follow. So, I just chat to candidates and employ those who I feel I and my people can work with. I never left it so late that we needed instant skills. To me that would be poor management.

Prior to that, I have tested and done many of the things people cite here, but most of the time I got awful employees who turned out to be good at doing interviews.

Yeah, a bit hippy dippy, but it worked for me.

What getting someone to solve small problems like FizzBuzz in the interview gives you an idea of how then approach the problem. Can they decompose the problem into smaller chunks ? Can they then refine their solution into a better solution ? Do they know when to stop refining before the solution becomes difficult to understand ? These are the skills you want to asses from a potential programmer.

Programming languages, platforms, frameworks come and go but if you don't know how to approach solving a problem then that knowledge is ultimately worthless.

Test-driving, however, is unfortunately neither scalable nor "leak proof". As soon as one candidate gives away your test drive question, the next candidate could easily cheat away inflating their apparent awesomeness compared to other better candidates.

However it would be incorrect to write down all whiteboarding interviews as "evil". Like any other interviewing techniques, it really depends on how you do it. A good whiteboarding question that allows candidate to use CS fundamentals to solve fairly non-trivial problem that you also needed to solve for your current work is a great question. There are 3 major reason why this doesn't work as expected at some large companies:

1. Interviewers can't think of good whiteboarding question and fall back to commonly asked popular puzzles. These interviewers are also often the ones who have one "favorite" question that they would ask everyone. This is absolutely #1 problem why whiteboarding interviews devolves in to secret puzzle marathon. The best questions that interviewer could ask are actually the ones that they needed to solve themselves as part of their work recently. This keeps questions relevant and refreshed regularly. It also allows to compare candidate with themselves and follow the philosophy of hiring people smarter than you. It's however hard because interviewers themselves needs to continue doing something interesting regularly.

2. Interviewers provide feedback to each other during the loop. It's not coincidence that if 1st interviewer gives "hire", all rest tend to do same at companies where there is no clear policy of not communicating feedback before end of the interview.

3. Interviewers don't follow up candidate response by extending question, building on to next level perhaps open ended scenario, asking auxiliary details such as test cases, complexity etc.

Next up in the topic cycle: Discrimination in tech. Why working more than 40 hours a week is killing your productivity.

I wouldn't want on critical infrastructure code alongside someone who can't reason through the performance implications of their code on the fly. You can waste a hell of a lot of time diagnosing and fixing performance problems because someone built an entire module around an inappropriate data structure.

At the end of the day, as Hal Abelson said, computer science is not about computers, or science. It's about being able to formalize process.

https://twitter.com/KentBeck/status/299528659746840576

I'm one of those who freeze up like a deer in headlights when asked to do a quiz. But have a degree in CS, have made several enterprise level packages by myself, and have been programming for 31 years -- so am fairly confident that I can program.

(If you need help posting a comment, feel free to use any of these samples: “You make todo lists, you don’t need real software engineers”, “Math is actually really important, you know!”, “Google is worth one gajillion dollars and they use quizzes, so there!”)

Nothing is said about how you actually get to this final list and that of course is where the real challenge lies.

Excellence in the grammar is also irrelevant.

I can understand the reasons why not to use algo problems as a filter, but I would have thought simpler forms of intelligence testing would still be quite useful.

Being born in the other side of the Atlantic if it wasn't for HN, I would never heard of FizzBuzz in my 14 years of software development.

https://europa.eu/epso/application/passport/quiz.cfm?lang=EN...

I don't mean this as flippantly as it might sound. I used to think it was kind of fun and exciting to be at the whiteboard, thinking on my feat, dealing with various curveball data structures and algorithms questions. I certainly always read up on this stuff prior to interviews, and from what I've heard, I'd be advised to do it again if I get an interview at google or something.

But the last time I did this left me depressed. I realized that I've done so much of my work in obscurity that people are still asking me about binary trees. I want to be very clear that I don't blame them! I also started to realize that I have a lot more to offer than my ability to traverse various tree structures, and that the interviewers seem unaware of this. But it's up to me to show them, not on them to just take me at my word.

The last time I went through a massive, all day technical interview, I didn't get the job, partly because I didn't review my data structures and algorithms book (again), but probably also because I came off as cranky and irritated. I was busy that week, probably should have put it off.

Just earlier that week, I had taken several clients from pharma and semiconductor companies through some beta testing of our software, refining the model, getting feedback on use, optimizing the code base so it could give answers more quickly. The company I was interviewing for created supply planning, forecasting, and inventory management software for various large businesses. As a math major with an MS in Industrial Engineering and a background writing software, it was right up my alley.

My interviewers showed no interest in anything other than my ability to code or a couple branches of math. In fact, some of them seemed so inexperienced that I'm not sure they were aware that this kind of experience even exists to be asked about (Oh, you work with clients? Our project managers do that). At lunch, one guy asked me how to swap two integers without creating a third integer. This is after a full morning of technical grilling on math and programming, with a lengthy afternoon to follow.

Fortunately, I'm now in a job where I am encouraged, not just allowed, to contribute to open source, do presentations at conferences, try to build communities, and so forth. I would not consider any job that did not have this element (and now that I have a job like this, I'm not looking anyway). But if I were to look around, I would vastly prefer to be hired for my known and widely used code base than for my ability to detect cycles in linked lists, implement a hashing function, or print out all possible permutations of a string (with no duplicates!).

I'm not there yet, but that's my career goal.

The day was broken up into sessions of about 1 hour each (maybe a bit more). In each session it was me and 3-4 other people, but the people rotated, so I got to meet lots of engineers, a PM or two, the hiring manager, etc., and they all got to meet me. A few people showed up more than once.

The first session was basically an introduction. I think it was shorter than the others. No quiz-style questions, but stuff about my background, etc., and opportunities for me to ask questions also.

Next I did a presentation at the whiteboard describing some project I'd worked on. They had told me this would be a part of the day, so I could have prepared (though I didn't). I talked about my recent startup, describing some of the data model and machine learning tricks I'd developed. They asked lots of questions, some to clarify, others about my reasoning, how it would scale, if I'd considered technology X, etc.

In the next session was me at the whiteboard again, but here they described a new feature they'd like to add to their site, and my job was to sketch out a rough solution. So partially this was about my ability to ask clarification questions, and also about my big-picture design skills. Again they asked lots of questions, and once we even got into a specific SQL query (where I opted to use an EXISTS with a correlated sub-query). So this session seemed a lot like the first, but more impromptu on my part and probably more practiced on theirs.

Then they took the whole team out to lunch at a nice Thai restaurant (~15 devs plus the director of engineering and the PM). This part was relaxing and fun. Of course this was still part of the interview, and wouldn't be relaxing for everyone, but still it was a good way to feel out personality fit for both sides. Also there were too many people for me to be the focus of attention, so that was a nice break.

Finally there was one afternoon session where they gave me a laptop with a toy Rails app, and had me add a feature (change a relationship from one-to-one to one-to-many, or something like that). There were two devs watching me code, and I had to do the whole MVC thing: write a migration, tweak the controller, edit the view. I got to talk through the changes as I did them, so they knew my reasoning. It made me smile to slip in a few fancy vim moves; I don't know if they noticed. But it made me realize this was also a test about some basic mechanics, too. They let me choose my editor/IDE, so it was a chance for them to watch whether I knew my tools.

Then there was a final session for follow-up questions etc. This was fairly short I believe.

All along the way I had lots of opportunities to ask questions of my own, which I appreciated.

There were no puzzles, no trivia quizzes. Personally I'd add a FizzBuzz test to the initial phone screen (like Han Solo "They hardly asked me any questions."), but otherwise I plan to completely rip off this template the next time I hire someone myself. Maybe it can help some of you!

>how little it had to do with the actual job.

That doesn't make any sense. How is solving a problem with javascript irrelevant to the job of a front end developer? That's what they do.

The best way to do it is pay the person per task as an independent contractor until you're sure a full time position is what you're both looking for. If they're helpful to you then you keep calling them and if not everybody keeps their dignity, no big deal. Don't humiliate potential employees right off the bat with stupid games.

I don't mind it. I've interviewed for enough jobs and had enough acceptances and rejections to learn something: it's not a big deal. If you don't know an API function, you don't know it. The likelihood that you won't get the job because you didn't know that one question is unlikely.

In fact, part of the evaluation is whether you can handle not knowing the answer with grace. The fail-outs are the ones who use their cool or seem to think the question is stupid, not the ones who get it wrong.

Once you realize that you don't need to get all of the answers to pass an interview, it gets easier.