Hacker News Reader

Comment by gvb | Hacker News Reader

[Interviewers] ask simple coding questions like the fizzbuzz, but [their] applicants still fail – even ones that shouldn't. [...] [Y]ou're using the wrong medium for what you're trying to achieve (e.g. coding over the phone).

Clap clap clap.

The same people that insist that the minimum development station is a tricked out computer with two (three?) big monitors, SSDs, the best IDE money can buy, etc. don't understand why someone is uncomfortable writing a program on a whiteboard or, even worse, over the phone.

Unfortunately, I don't have the solution. Perhaps pg can add "Interviewing that doesn't suck" to his list http://ycombinator.com/ideas.html - obviously this is included in "7. Something your company needs that doesn't exist." and is the company version of "8. Dating."

Personally, I think Steve Newcomb has a 90% (maybe better) solution. http://blognewcomb.squarespace.com/essays/2010/10/14/cult-cr... (discussed here http://news.ycombinator.com/item?id=1793095).

View on HN

My company does whiteboard interviews. The important thing is not getting the right answer. We don't bother to attempt to compile the applicants' code, so we don't even know if it works. The idea is to get a glimpse of how they reason through a problem. After they've roughed up a solution we ask them follow-on questions like, "How would you test this? What corner cases would you check?" If it suddenly dawns on them that they forgot about certain corner cases, that's considered positively; if they blunder confidently on through without checking to see if what they'd written would really work, that's considered negatively.

We also ask them a design question, typically something like "design an automatic retrieval system for one of our robots". Again, not being mechanical engineers we have no way of telling whether their ideas would actually work. It's the thought process we're interested in looking at: how detailed they can get, how they anticipate and solve or work around problems with their design, etc.

Given that we have a pretty top-notch software team, I'd say our system has worked out rather well. :)

The problem with the Newcomb solution is that it is aimed at the high-end of tech companies, which are ultimately willing to pay very high portion of the company worth for the top talent.

Most companies employing programmers today are not such companies -- they cannot possibly hope to hire A+ employees, but they are trying to build a solid team of B- or C-level programmers, just hoping to keep the F-level out. As far as I know, this is a total crapshoot and the ones succeeding in it do so mostly by luck.

Understood and agreed. I thought about editing my comment to add a disclaimer about "if you can implement it" because many companies cannot.

On the other hand, the same companies that are decrying "99 out of 100 cannot program" also claim to be in the same high-end tech company space as Newcomb.

One of the most eye-opening lessons of my professional life:

I had a set of coding questions to give final round candidates to complete in person. By paper, whiteboard, or laptop -- their pick.

The success rate for candidates was about 25%.

I then started giving the same questions as a time-limited take-home test, for first-round candidates.

The success rate jumped to almost 100%.

Same questions. Less-filtered candidates. The difference? The primary determinant of success was interview environment, not candidate ability.

That's exactly what happened to me in my recent interviews.

Some company wanted me to write a simple string reversion and prime number finder programs on paper, I failed because I'm not used to code on paper.

Some other company wanted me to write a server client program which takes a file from client and saves it to server. I completed it on my laptop on the same day when I got home.

If the interviewer is going to make the interviewee take a written test; it must be about concepts, designs and stuff, not implementation.

Out of curiosity was the failure at an algorithm level or at a semantic level ? (i.e. could you not put the algorithm down on paper, or was it just a case of not remembering the correct syntax or libraries)

Algorithm was fine in both cases, I had the problem solved in my head but couldn't pass it onto paper effectively. I had trouble remembering semantics. Because I use autocomplete functions of IDEs a lot.

Personally, (visual and diagnosed ADD) it's very difficult for me to think through algorithms under pressure, or in environments with distractions. Stuff that takes me five minutes in a quiet, private, relaxed setting can take me an hour to think through in an environment with more people. Lab courses in college were particularly frustrating for me for this reason.

Why would anyone ask to find primes? Most people don't know the best algorithm and anyone who needs to code efficient ways of determining primality has intense number theory training.

The "naive" solution typically is a sieve method and checks more than the required number of factors (goes beyond sqrt(n)). It's like asking someone to sort an array and expecting bubble sort because you never use sorting algorithms in what you actually code.

The best way I know (and there are actually better) to find primes is using an approximation algorithm that no one would ever come up with independently without months of research. The Miller-Rabin primality test finds if a candidate for primality n has a % chance of being prime by looking for numbers a < n that don't satisfy Fermat's little equation. Almost every composite number violates Fermat's little equation (Carmichael numbers are the only exceptions).

Once you hit the desired probability that n is prime (testing more a < n raises this probability) you use a deterministic method to determine 100% if n is prime. If it isn't you throw it out and use Miller-Rabin to find another candidate. Even though you might throw some candidates out and have to pick others to check for primality this method is far faster than a purely deterministic solution using a naive sieve methods.

This is a moronic test of ability. It exposes the arrogance of the company in assuming they can evaluate candidates in 15 minutes by asking the most challenging number theory problems out there when they have no understanding of the current methods in the discipline.

The goal is not to illustrate the best algorithm known to researchers, but simply to demonstrate that the candidate can explain the obvious algorithm and get reasonably close to correctly implementing it. Primes is an easy problem to explain and to solve (the naive way), but I wouldn't ask such a commonplace problem because a sub-par candidate may have faced it before and partly memorized the answer.

It's not the best algorithm known to researchers. It's the best algorithm known to people who took any course in number theory or a higher level computer science course that covered approximation algorithms. This is used in actual applications like crypto (RSA) frequently.

It is exactly like asking an applicant to sort an array and expecting the naive bubble sort response to check they can develop and implement a naive algorithm for searching. It says you can brute force a solution with a for loop. It says nothing about how well you understand advanced data structures, recursion, algorithmic analysis, language features, domain specific knowledge, program design and structure, or knowing when you you're out of your league and should google the best approach. It quizzes chapter 3 of any intro programming book and basic division in a situation where you would never use it.

Ask them about event handling or something useful like hash tables or linked lists or version control systems. Any math major can answer this question easily without knowing much beyond for loops and arrays. It tells you almost nothing about their programming skills. I was math/cs with grad school in math and every math major I know could do this. Most of them have no idea how to code. I know some great developers who would struggle and fail because they typically don't use this math in their job and have forgotten it. They work at some of the top companies out there (which are frequently discussed on this webpage).

Or possibly the ability to cheat...

It's becoming more of a problem, especially with university level recruitment, that candidates are either getting others to take the test for them or downloading solutions from somewhere.

Often candidates once bought in for interview fail to be able to explain how code "they wrote" works.

I once interviewed someone who said they wrote a web browser in Qt. After inquiring a bit it turns out they had really forked the Arora browser I wrote. I had fun for several minutes asking him questions about the code that he clearly had no clue about.

This doesn't solve the problem for everyone else, but when interviewing, solve the problem for yourself: Bring your own laptop. (Or borrow one if you need to!)

I can't believe in 2010 this isn't "conventional wisdom", but heck, take advantage of the competitive advantage while you can. No matter how impressive your linked list reversal is on the whiteboard, running code is even more impressive.

(blush) That didn't occur to me. Good thing I haven't had any "program on a whiteboard" interviews. Thanks for giving up your competitive advantage to all the HNers. :-)

Personally, I hate to program on someone else's computer. I run pretty bog-standard linux with minimal plugins and aliases (in part, so that I'm comfortable in a bog-standard environment typical of servers). The problem is, lots of people have customized the crap out of their system. The result is added discomfort when I try to do something on their computer.

Someone once brought in his laptop. He showed me a running program and gave me a tour of his code. It was very impressive.

That's very much like what I would ask from a candidate developer: "Bring your laptop and show me some stuff you made in the past year(s), and expect to implement a small programming problem I give you. You may use your favourite development environment for that."

What happens if they don't have a laptop?

Wouldn't that be some kind of a contra-indication? Not a definite "don't hire" signal, but pretty close.

Now, sorry to say, but that's just a stupid presumption. It amounts to either economic discrimination against people who can't afford a laptop (for a whole host of possible reasons other than "I guess they never had a job that paid well because they are crap programmers"), or at the very least, prejudice against people who work on desktops and don't like laptops. Not every programmer is a laptop-wielding RoR nomad who writes code wherever, whenever. Some people need their special cave and their fixed rig.

Don't assume everyone is like you.

Given last.fms database of (user, (artist, track), datetime) plays, how would you quickly recommend me something to listen to? Now you have two problems. First, your code, in order to run, needs to load in and process this list, and you need to decide if it's a flat text file (the format of which you then need to define and parse) or if it is in a database (the connection to and schema of you will then have to deal with). That's boilerplate that's (a) easy to get around on the whiteboard (b) utterly irrelevant to the question asked. Second, you don't actually have such a list, so running your code isn't actually impressive. Mocking up a sample list takes time, and engineering it to create useful results even more.

Next, I ask you to parallelise it, and you don't recall the pragma that OpenMP uses, so you just need to pull up the manpage, or you don't have the Actors library installed, so, uh, what's the password for wireless? Oh, and you still don't have that large file, so even if you get your parallel code running before I run out of patience, waiting for the answer "well, access to this counter would have to be synchronised, so that would create a bottleneck", running your program will still not have enough data to munch on to gain any insight in it's performance.

tl;dr: You will have to be very sure of yourself to pull this off. I want to know how you think, not how you handle your tools.

Yes, much better to just handwave; that'll show you have it all together.

You're seriously arguing against having somebody actually code in their interview? I'm not saying it has to be the entire interview, but how incredibly crazy is it to actively argue against the idea that maybe you, as the interviewee, might want to actually show you can do what you say you can do?

Besides, I find myself questioning if you've been on the interviewer side of very many interviews. Finding people who can slurp a file in at all is a damn good start, unfortunately.

Then again, I know I do this really whacky thing where my interview isn't set in stone, but I actually react to the skills demonstrated on the other end of the table. Believe me, if somebody walks in sporting a laptop I'll customize something up for them to show me out of my base set of questions. It would hardly be any different than what I do anyhow.

Lay off the strawman, whiteboard programming is not handwaving.

And I can do without the ad hominem as well. No, I haven't done a lot of interviews (about 6-7 I think), but my post still has a substance that can be argued against.

> You're seriously arguing against having somebody actually code in their interview?

No, I have no problem with coding in an interview (I think it should be part of it, just not the part where you do something else). I have a problem with generalizing the advice "When asked to do whiteboard programming, whip out your laptop.".

The person who is interviewed should be interviewed at least for a full day. They should be presented with a large set of problems, both "fizzbuzz" type, design problems, debugging problems. They should talk to the top technical people in the company, with those they will be working with, and with as many other people as possible.

This is even more important in a small company. A small company cannot afford to hire a dead weight. A couple of those will kill the company -- they waste resources, don't get work done, and, what is perhaps very dangerous, drive away top technical talent who does not tolerate mediocrity.

This sounds great in theory, but can also be a big time waster for your top dev talent. Especially so considering you usually start hiring when you're especially crunched for developer resources.

The interview process should be extensive, yes, but it should also be spread out in a way that allows you to fail candidates early. Blocking out a day of interview time at once can be inefficient - rarely will that person be sent home early (for various reasons) and they are now just wasting a bunch of peoples time, and their own.

The critical people interview first, after the interview they talk to one of the owners. At that point the interview could stop. So it might actually not take more than half an hour.

Hiring many people at one time I think is a sign of a problem. We try not to hire more than 3-5 people / year. It is not that bad for a each top dev to spend 10 hours / year to find the right candidate.

Think of all the time they will be wasting with this person if she turns out not to be a good fit.

You presumably want people who will enhance and help your top talent.

That's exactly the point and this inversely related to the size of the company. Imagine if IBM hires a dead weight, the company is so big and there are so many other positions that the company as a whole will not be terribly hurt (unless it's the CEO perhaps).

But with a small company, when you have 5 people and you hire the 6 and it turn out to be a bad choice, it could kill the company.

I don't understand this idea that it is a good idea to waste such vast quantities of time for both the potential employee and the employer. Whenever I've looked for a job in the past, I'm typically still employed. Using PTO to go to day long interviews means I can't give a company a shot unless I already know I want to work there.

Because when a small company is hiring an employee they are risking quite a bit. It is worth spending more time finding out who they are. Hiring and firing people immediately because they turned out to be the wrong person is very counter-productive and demoralizing for everyone involved.

Before the day long interview we would have a phone interview. At that point you would decide whether you want to take a day out of your life to find out more about a company where will potentially spend years working. I don't think that is that unreasonable?

It depends. What is the likelihood that that person will get hired if they go on the day-long excursion? Will this person be just one of many people you will take on these marathon interviews? Or will you be completely focused on this one person, and if they seem like a good fit they get hired immediately?

The point is, if you expect me to invest so much in an "interview" with you, you had better be laying some big chips on the table as well.

You will pretty much be hired at the end of the day. We hire about 10% of people that we interview this way. So we already pre-screen people before spending all that time with them.

Remember you can always leave and we can always stop the interview.

Well this way of interviewing has worked well for 15+ years ;-) , we like it, we'll probably stick to it.

A 10% shot at most jobs is not worth 1 day of PTO.

It is not that random. When I went for the interview I was 80% sure I would get it. But I wanted to spend the day there to confirm that. I was interviewing them as much as they interviewed me. I had multiple job offers and meeting the people I would work with helped me decide.

After being there for 4 years, I would it was the best 1 day of PTO ever spent in my life.

>> The same people ... don't understand why someone is uncomfortable writing a program on a whiteboard or, even worse, over the phone

Exactly. I loathe whiteboard interview questions. In five years of programming, how many times have I ever written or looked at code on a whiteboard to solve a real, work problem?

Zero.

Yet some people think the best way to learn about a candidate is to put them in a situation far removed from a real, daily work situation and see how they perform.

Full working code on a whiteboard, is very uncommon. However a lot of the time useful code snippets go on a whiteboard. Particularly in informal meetings. More than that, design diagrams go up there as well. There are some interview questions that I like which force people to put up the sort of pictures we actually draw in meetings - for instance come up with the tables backing a particular application. At the place where we asked that, you'd find yourself doing that after you got hired as well.

If you don't use whiteboards in these ways, then I'm going to suggest putting whiteboards near developers and consciously try it. You'll be amazed how well it works.

The general principle, as I understand it, is that if you can't write a very simple program like FizzBuzz under pressure, you will also do poorly when you're doing harder tasks with less pressure. This aligns pretty well with my experiences, both in school and industry.

Really, if you can't write a few lines "on the spot", what use are you going to be to an employer? In the real world there is always some deadline. I don't always expect perfection in a whiteboard scenario, but there's a world of difference between a blank stare and a 90% correct solution with an off-by-one.

I disagree. Give me a hard problem and a short deadline, leave me in front of my computer with my environment set up (so I can do some quick trial-error), don't look over my shoulder to evaluate me (but you can help me if we're doing pp), and I'll succeed. Give me an easy problem over the phone (I'm a visual) or on a whiteboard (my handwriting is not good) while timing me, and I will fail.

But in the end, I like this kind of interview, because I don't want to work for anyone who would evaluate me like that.

The other thing is that these interviews almost always enforce no Internet access. If I was in a position to hire, I'm looking for people who can solve problems, not those who can remember the Java API in their head. If their first instinct is to open Javadoc, then my reaction should be "good, they know where to look for to make sure it's correct" not "God, why can't they just do this [arbitrary task]?"

I'm hoping to finish my PhD and become a professor in Computer Science. My plan, given university constraints, is to never once offer a written final. I've not once taken an exam where I was confident just writing code down. It doesn't make sense. Offer them a final on a computer with Internet access, but with a sufficiently difficult problem that it can't just be Googled for a complete answer. Their web access logs are turned in with their exam, so I can see if they copied something if it looks suspect, or if they're borderline I can give extra credit for going through a mental process that shows promise.

"If I was in a position to hire, I'm looking for people who can solve problems, not those who can remember the Java API in their head"

That is what makes the whiteboard such a good tool: when working on a whiteboard, you can focus on the problem, not on pesky details. A whiteboard will never complain about API issues, function naming, etc. Not even by using squiggly underlines.

I agree and think people are getting caught up with writing complete running code on a whiteboard which should not be what the whiteboard is for. A whiteboard is for writing pseudocode to quickly build algorithms, work on design or solve other 'thinking' type problems. It's not for testing if someone knows obscure syntax issues (there are actual tests for that!).

In school we were taught to work through algorithms and other design using pseudocode and pencil and paper. Doing this also helped reenforce that languages are very interchangeable using the same algorithms and designs. I wonder if that is simply not done anymore in school?

I think another way to impress an interviewer is to come to your interview prepared to discuss (with technical detail) past exploits in which you were able to do just what you described. Most people just flow with the interview rather than try to (subtly) direct it in a way that is more favorable to them. This is a workaround for sure, but aren't programmers used to finding workarounds to impossible situations?

Another option is to tackle the interview itself as a challenge and practice interview skills, such as whiteboard coding. I think Steve Yegge suggested this in one of this blog posts about how to interview at Google. Personally, I find this to be very impressive: You knew there was going to be whiteboard coding, so instead of complaining about it, you practiced it. If you (casually) mentioned this during one of my interviews, I would probably give you some extra points.

Deadline pressure != interview pressure.

Also stressful deadlines are an indicator that sosmthing else is wrong in a company. The company should fix that issue instead of hiring people that can "program under pressure".

At my company (we make AUVs), stressful deadlines mean you're out on a boat and you've got ops personnel looking over your shoulder and if you can't push out that code fix or configuration change NOW then they may not be able to run missions for the rest of the day because the vehicle would be dead in the water and the sun is going down.

Needless to say, actual software development in the office is pretty relaxed. But when you go out on ops, you will be in the pressure cooker and you will need to handle it.

Plenty of code changes need to be made under pressure. Some recent examples from stories posted on HN:

1) Bug in Google authentication which allows websites running on App Engine to grab users email addresses without permission

2) Crunchbase making all historical searches public

3) Facebook downtime bringing down "like" buttons

All three of these issues were for the companies in questions bugs that had to be fixed in realtime. Does it indicate that something is wrong with these companies ? - no.

Any changing production system is going to suffer from bugs that on occasion will require programmers to be woken up in the middle of the night to urgently fix a bug.

"Does it indicate that something is wrong with these companies"

It means that their testing wasn't very good.

If programmers are being woken up to fix something then something is very wrong with an environment.

You're never going to have 100% test coverage. Any complex system is going to have bugs. It's pretty much impossible to write even a small library without bugs, let alone huge websites with the complexity of Facebook or Google.

Look at cryptographic libraries. Often they've been gone over line-by-line by hundreds of experts in the field, have extensive testing both unit and field. Yet bugs still get found in them.

My point isn't that bugs happen - of course they do and of course they get missed by tests. It is that if you pull someone out of bed to fix a problem I would suspect there is a high likelihood that you will be introducing more problems when you deploy this fix unless you run a full QA cycle after the change.

don't know about 1. and 2. The 3rd - Facebook - from what i read about it (bug in cache invalidation/update, herding the db) as well as recent problem at Foursquare (uneven sharding on MongoDb overloading a shard), as well as the production problems i have faced and resolved personally, have nothing in common with interview questions i've been asked so far in my life. Not even close, not even in the same Universe.

The problem is that real world bugs are often too complex to use in interview questions. From my experience solving production bugs often involves having to piece together bits of information from logs, databases, etc. and making educated guess to track down the source of the bug. It's pretty hard to replicate in an interview environment.

Spotting bugs in code, operational questions, and writing code samples are probably the closest you'll be able to come within the time restrictions of an interview.

Occasionally I have to troubleshoot our production code while we lose several thousand dollars per hour in revenue and jeopardize business partnerships. Sometimes it's just pointing out which servers need a kick in the head, but sometimes it's a recent code change that didn't handle all cases and we need to fix forward on the spot (because rolling back all interdependencies would take longer).

You should be able to reason regardless of how big or interactive the surface you're messing around with is.

The reason that big monitors are useful isn't because it makes you smarter or more able to reason about trivial problems. It's because it provides more space to cross reference code, more space to look up documentation, and so on.

For the sorts of questions that are being asked in an interview, you should be able to sketch out a solution on a whiteboard no problem.

Reply on news.ycombinator.com