How to Interview Engineers | Hacker News Reader

How to Interview Engineers

blog.triplebyte.com

468 pointsFabioFleitas475 comments

The sad reality of programming interviews is that it's absolutely necessary to ask several near-trivial questions in order to flush out the candidates with awesome resumes and impressive degrees who simply have no idea how to analyze a simple problem and solve it using a computer.

Lately, I've been asking "given the starting and ending times of two calendar appointments, determine whether or not they conflict." No loops, no fancy algorithms, no tricks... and asking it has not been a waste of time. I get people writing doubly-nested loops over all of the seconds in the two intervals, comparing for equality; I get multi-line predicates full of redundancy that still yield false positives and negatives; it's depressing as hell.

The funny part is I reckon you would struggle to answer the question yourself when put on the spot, if you had no prior experience of it. Just my opinion of course and I am speculating.

What's your basis for that speculation?

The solution is a simple predicate. Assuming both events are internally consistent (i.e. end time after beginning time) they won't overlap so long as the beginning of the first comes after the end of the second OR the beginning of the second comes after the end of the first.

Seems like a good weed-out problem.

Disclaimer: no prior experience with the problem.

I think many companies hire with the number one goal of reducing false positives. By definition this approach is going to unfairly reject a number of candidates. I would argue that it also dehumanizes a number of candidates, forcing them in aggregate to play a numbers game until they are in the top 20%-40% of the pool (if they ever get there) where companies then begin vying for them.

It is as you put: depressing as hell. And this is among white collared, educated, demographics. Imagine how it must feel to be an uneducated demographic in most other parts of the world...

> By definition this approach is going to unfairly reject a number of candidates.

This is only a problem if the interview/hiring/capitalism process is meant to be fair.

> Imagine how it must feel to be an uneducated demographic in most other parts of the world

The less privileged here certainly had jobs in HS and college where you just filled out an application, the manager made sure you weren't a convict or on drugs, and you got the job. The "uneducated demographic in most other parts of the world" is probably not solving algorithm quiz questions on a whiteboard.

> This is only a problem if the interview/hiring/capitalism process is meant to be fair.

It's also a problem when those same companies want to bitch and whine about a "talent shortage" or a lack of diverse candidates.

> It's also a problem when those same companies want to bitch and whine about a "talent shortage" or a lack of diverse candidates

so much this. Why do companies get to receive all sorts of incentives to hire when candidates they reject walk into other great tech firms, make bank, and generate boatloads of value?

Maybe some should just pay more, too. There are plenty of firms just being too cheap to hire great talent, including some large, well-known ones.

I'm not a programmer but wouldn't

if (endtime[1] > starttime[2]){status=conflict}

work? Assuming time is encoded in epoch format.

And assuming that "1" and "2" refer to the appoints in order of start time, but they may not be given in that order.

also if (endtime[2] > starttime[1]){status=conflict}

That's half of it, but doesn't account for the case when starttime[1] is later than endtime[2] and they don't intersect.

Good point. I didn't think about them not being written down in the list in chronological order by start time.

It works if you assume that appointment 1 starts before appointment 2 starts (or if you explicitly sort them by start time).

e.g. in J:

       'a b c' =: 50 60 3 4;3 50 49 99;2 10 10 12  NB. 3 different sets of appointments
       3 :'ok`nope{~0>*./-~//./:~_2]\ y' every a;b;c
    ┌──┬────┬──┐
    │ok│nope│ok│
    └──┴────┴──┘

This is close but you need to ensure the start time is before the other ends, and test for the second one starting before the first also. There's four cases:

[appointment 1 start] [1 end] <-- some time --> [appointment 2 start] [2 end] (case 1 - no overlap, appointment 1 first)

[appointment 1 start] [appointment 2 start] <-- some time --> [1 end] [2 end] (case 2 - overlap, appointment 1 first)

[appointment 2 start] [appointment 1 start] <-- some time --> [2 end] [1 end] (case 3 - overlap, appointment 2 first)

[appointment 2 start] [2 end] <-- some time --> [appointment 1 start] [1 end] (case 4 - no overlap, appointment 2 first)

So you need to do:

if (appointment1.end > appointment2.start AND appointment1.start < appointment2.end) OR (appointment2.end > appointment1.start AND appointment2.start < appointment1.end) // conflict

> if appointment1.end > appointment2.start OR appointment2.end > appointment1.start // conflict

This actually isn't correct. Consider the events (0, 3) and (4, 6). The end of the second (6) comes after the beginning of the first (0), but they don't conflict. You want `and`, not `or`.

EDIT: oops, just saw you edited it. Good catch :)

Your cases don't consider [1 start][2 start][2 end][1 end] or the opposite, but your pseudocode still covers those cases.

True, I didn't include those cases as they are not unique in terms of why there is / isn't a conflict. But you're right, worth including for completion.

> Your cases don't consider [1 start][2 start][2 end][1 end] or the opposite [...]

Your notation there, where instead of a pair of start/end pairs you have a list of tagged times, reminds me of a good approach if one is doing a generalized version of the problem: given a list of N appointments, find conflicts.

Make a list of tagged times, where a tagged time is a triplet (time, 1, name) if appointment named "name" starts at time "time", and is (time, -1, name) if appointment named "name" ends at "time".

Sort the tagged time list with time ascending as the primary sort key, and the start/stop tag ascending as the secondary key.

Now to find conflicts you simply scan through the tagged times list, keeping a running total of the start/end tag values. If the running total is greater than 0 when you begin to process a given entry, that entry has a conflict with an earlier appointment, and the running total is how many earlier appointments it conflicts with.

As described above, this lets you print a list of what appointments have conflicts with earlier appointments, but it doesn't give an easy way to say which earlier appointments conflict. If you want to do that, it is straightforward. Just add a set data structure, and during the scan of tagged times add "name" to the set when you encounter an appointment's start, and remove "name" when you encounter an appointment's end. When you find a conflict, the set contains the names of all of the earlier appointments the present appointment conflicts with.

The above assumed that two appointments do not conflict if the ending time of the first is the same as the starting time of the second. If that should be counted as a conflict, just change the sort so that the secondary key is sorted descending instead of ascending.

Does anyone really test for specifics like this? I thought the goal was to ensure the candidate understood that dates and times are abstract numbers, anyone that applies a mathematical operator should pass it. Yes this would still filter out a lot of people.

Or just sort by start time, then do the simple check.

Yeah, this is way easy if the two events are sorted by start_time first.

I was thinking along the same lines as you, but it turns out you only need two comparisons and one logical operator.

barrkel has a thorough explanation here:

https://news.ycombinator.com/item?id=14641485

Or, as I realized later, it seems helpful to me if I look at it as a practical problem instead of an abstract one:

https://news.ycombinator.com/item?id=14642059

  a = Appointment 1 start
  A = Appointment 1 end
  b = Appointment 2 start
  B = Appointment 2 end

The only predicates are: a < A, b < B

The complete set of orderings are thus:

  aAbB - no overlap
  bBaA - no overlap
  abAB - partial overlap
  baBA - partial overlap
  abBA - complete overlap of one appointment inside the other
  baAB - complete overlap of one appointment inside the other

To find whether a and b intervals overlap (ends are not included):

  overlap = a.start < b.end and b.start < a.end

Why is it downvoted? The formula is correct and it is simple -- if you don't understand it, just ask.

I'd say so. The "assuming time encoded"-bit is important though - I rather think starting with the data is valuable:

Can I assume the times are in a sane date format/datatype? If not, start with e1 = toSaneDateFormat(endtime1), s2 = toSaneDateFormat(startime2) (Fill in if interviewer is interested).

Then, as you say, a check for overlap is easy - but maybe one wants to be more fancy, like: if (timeDelta(e1, t2) < timeToWalkFromAtoB, or < 5 minutes -- they should be considered an overlap? If they are on different continents, maybe < 24 hours should be an overlap?

At any rate, I'm guessing (hoping) this leads to discussions about representing dates, and what the business logic is (eg: physical meetings - you can't teleport from one location to another).

[ed: And as others have touched on, if you deal with timestamps/raw number types - be careful that you don't end up with appointments in "wrong" order - I'd say a sort() aware of date-objects might be your friend here.

ed2: In fact, if you can assume a sane date-type, and timeDelta, you could probably assume an "interval" type, and simply ask for overlap?(appointment1, appointment2) ... ]

This is a great example of why the OPs question could be a good or terrible interview question -- it's not clear if they want the obvious technical solution (comparing datetimes) or a wider discussion about "what is a date time and how is the data represented", "what are the real world / business implications", "here is existing technology using intervals that will solve it" etc as you mentioned.

In my experience interviewers are usually looking for the technical solution despite the business oriented solution usually being much more applicable (and thus relevant) in the day to day role.

Key thing to remember here as an interview candidate is to clarify with the interviewer the scope of the question and the nature of the answer they're looking for. If for instance the interviewer starts with the simple technical solution and then probes the business aspects this might be a nicely rounded question.

My company asks a question during engineering interviews that is overly simple, but a bit vaguely defined on purpose. The main objective is to see if they can ask clarification questions to determine precise requirements, which is something that every engineer does daily on the job.

> given the starting and ending times of two calendar appointments, determine whether or not they conflict.

Say the first appointment goes from a to b and the second goes from c to d. What does it look like if the appointments _don't_ conflict? This happens just when one appointment ends no later than the other begins; in other words, when b ≤ c or d ≤ a. So, the appointments conflict just when not(b ≤ c or d ≤ a). We can then distribute the negation using [De Morgan's laws](https://en.wikipedia.org/wiki/Negation#Distributivity) to arrive at this: the appointments conflict just when b > c and a > d.

Consider appointments from 1-3 and 2-4:

a = 1, b = 3, c = 2, d = 4.

Do they conflict?

It's difficult to tell if GP miswrote, or you misunderstood, but it doesn't sound like they've made the mistake you're alluding to (I would expect them to be explicitly wrong).

Hmm.. Perhaps I did misunderstand something. But if the test for a conflict is:

b > c and a > d

with conflicting appointments from 1-3 and 2-4:

a = 1, b = 3, c = 2, d = 4

won't that evaluate to:

3 > 2 and 1 > 4

true and false

false (no conflict)

I could be wrong, but I think you have to do all four comparisons as in aldarn's comment:

https://news.ycombinator.com/item?id=14641043

(no fair peeking!)

Yes, I do not think "and" here was meant as a logical constraint.

The test is the opposite of (X ends before Y starts) and (Y ends before X starts). You only need two comparisons.

With two events, X going from xb to xe, and Y going from yb to ye, you have these possible sequences:

    xb xe yb ye (no conflict)
    xb yb xe ye (conflict)
    xb yb ye xe (conflict)
    yb ye xb xe (no conflict)
    yb xb ye xe (conflict)
    yb xb xe ye (conflict)

As long as (xb >= ye or yb >= xe) there's no conflict. You can flip those: (xb < ye and yb < xe) to get the conflict condition.

We can evaluate this expression over those cases:

    xb xe yb ye : yb < xe => false => no conflict
    xb yb xe ye : xb < ye and yb < xe => true, conflict
    xb yb ye xe : xb < ye and yb < xe => true, conflict
    yb ye xb xe : xb < ye => false => no conflict
    yb xb ye xe : xb < ye and yb < xe => true, conflict
    yb xb xe ye : xb < ye and yb < xe => true, conflict

So you only need two comparisons, and an and operation.

Yeah, you have it right. I had a nagging feeling that two comparisons ought to be enough, and something was just reversed in bumbledraven's formula. My first thought was that the 'and' should have been an 'or', but that was wrong. It was the comparisons that were reversed, as you demonstrate.

Just to compare with the previous comments, if we put your notation back into the a-b c-d format, then it would be:

conflict = c < b and a < d

It's interesting to note how the choice of names makes such a difference. With the arbitrary names a, b, c, d for the four times, it's harder to think about whether the expression is right. Which was 'c' again?

Your names xb, xe, yb, ye are still terse, but once you know that x means one appointment and y means the other, and be and e mean beginning and end, it makes it much easier to think about it.

Just make sure your data is correct, and a single comparison works: https://gist.github.com/anonymous/c02d74eb75a51dce8dcbc027a3...

Your code has two comparisons; one to figure out which one starts first, and another to figure out whether the first one ends before the second one starts. This isn't getting your data correct; knowing which one comes first isn't part of the problem description :)

It appears what they're alluding to is this part: > the appointments conflict just when b > c and a > d.

which I first understood to be a conversational statement rather than a statement of logic (which I assume GP is referring to), in which case it should be (b > c or a > d).

You have a mistake in the last not of your formula. The opposite of d<=a is d>a, not a>d.

This mistake could be made more "eyesoreous" and possibly avoided by naming the begin/end times b1, b2, e1, e2 instead of opaque letters. I would fire the OP for that alone :p

do you mean "b > c and d > a"?

If you were writing code, I would argue against using De Morgan's laws just because you can.

You're saving a few characters to make the code less obvious. Please correct me if there's some other benefit.

Usually after a mathematical simplification like this, you can find a direct explanation for it. In this case you can reason:

If two appointments conflict, that means that there is a time when they are both active. Thus each of their end times are after both of their start times. Of course, each appointment ends after it started, so that just leaves that each appointment ends after the other started. So if the appointments are a to b and c to d, then they conflict iff b > c and d > a.

(I've corrected bumbledraven's mistake of a > d vs. d > a.)

I would say, if the transformation makes the code and the reasoning behind it simpler then do it. Otherwise, know that any decent optimizing compiler can do this trick by itself so what you write only influences readability.

If one appointment is wholly contained in the other, then b > c and a < d.

If the candidate starts by drawing 2 lines, then marking start and end on the lines, you know you would be off to having a good chat

My thoughts exactly. Defining the problem space and all possible cases is the key that so many people miss. Once you do that, the solution is just Boolean logic.

To have a good chat I would hope they drew a coordinate system, mapped both appointments in Euclidian space, and then did an intersect between vector A (first appointment) and vector B (second appointment) to see if there are any data points in common.

What does this mean?

For a good discussion, instead of looking at the problem strictly programmatically, you could look at it as something you could map in vector space.

The dimensions of appointments are [startTime, endTime, location]. If you have points in one appointment that intersect with other appointments, then you have a collision.

This problem is solved with a couple of simple if statements. Finding a "solution" that is more complex than necessary or introduces knowledge that really isn't needed is the sign of a programmer who... over-engineers and is likely to be a detriment.

It's actually a tricky problem to think clearly about unless you start enumerating the cases, or get pen and paper out to draw some boxes. I don't think it's completely trivial to verbalise. But I also don't think a competent programmer should fail to work it out.

I like the problem, and I'll probably use it in future :) It'll be a low bar for those times where you're getting the impression that a bigger task will be beyond the candidate, and you want to cut your losses early.

FWIW, I've seen similar complete failures to reason when applied to the even simpler problem of testing to see if a point is inside a rectangle. Some people seem to have a very underdeveloped one dimensional measure function in their brains.

It may be easier to verbalize than we think. I just realized that for me the trick might be to not be too abstract and formal when first thinking it out, but look at it more like an everyday problem.

Instead of this:

"Given two appointments, one that starts at time A and ends at time B and another that starts at time C and ends at time D, how can I tell if they conflict?"

More like this:

"I need to run another errand while I'm in town for the interview. I can either do it after the interview, or if I do it first, I'd better be done before the interview."

That gives me something concrete to think about so I can make sure I am on the right track and won't miss my interview! And from there it's a pretty easy path to working out the detailed formula. If I jump into the abstract description right away I'm a lot more likely to confuse myself.

Funny; for me, the abstract version is exactly what I need. And the answer pops out almost immediately:

  B <= C or D <= A

(Assuming A <= B and C <= D, for which I may ask the interviewer about.)

Depending on what you're hiring for, it's really not all that depressing.

If someone has been doing maintenance programming in a company without excellent culture that values code quality - their brains start to rot pretty quickly.

Five years later, they know how to debug, estimate, work on a team, just not really develop anything from scratch or think critically.

If you want people to do real development, those are hard to come by. If what you really need is someone to maintain some bloated codebase and make it worse, but not too fast, then most people will do just fine.

Another thing is: what do you really want from a candidate? What led you to start asking them trivia questions in the first place?

The really good ones will be put off by trivia because it brings them zero value. If you ask questions that actually pertain to the field you're hiring for, it'll go over much better.

For example I applied to work at a major bank, and got asked a bunch of security questions, which exposed me not understanding how main in the middle attacks worked. The interviewee had requested that I go home, do a bit of research and email them the answer.

This brought them value because they found out that's an area I have little knowledge in, and it provided me with value because it pointed me in the right place to go research.

Trivia puzzles don't add value to anyone. Frankly if you're doing trivia and are upset by the response you're getting, a part of that is your fault :)

Making sense of his codebase is harder then writing from scratch. Maintenance programmIng requires a lot of puzzle solving. What it does not require is modern technology. E.g. maintenance programmer should answer this question easily and have problem it you ask about react.

You don't want to work with people who can't solve trivia like this. Something like man in the middle can be learned, basic problem solving takes time to develop.

Determining if two intervals overlap is not a trivia question but a trivial question for a programmer and hiring people who can't do better than O(n²) probably is good only for really boring maintenance projects where the workload lags far behind Moore's law.

I'm not necessarily saying that such things don't exist, but not my cup of tea.

What do you think n here is when you say O(n^2)?

The number of seconds in these intervals, see the top level post.

TIL building products that make money (as opposed to pre-revenue VC-funded CRUD apps) with a team that enjoys their job and can be home by 4 every day is not "real development."

Not all teams that do what you suggest write garbage code as the GP implied. I have worked a places with large garbage and excellent code bases.

The places that have garbage and years of technical expect far less from their devs because they can get far less. These are also the places that prevent tickets for refactoring and never let devs pay down technical debt, then they get mad when development is slow. They value the next piece of "functionality", even if it as simple as new entry in a menu more than they value their future. When there is so much technical debt that adding that menu entry takes a skilled developer 60 hours this becomes the norm instead of a problem to solve as a business.

I would rather not hire a developer familiar with such situations and failing to fix them. Whether the failure was their ability to recognize or communicate the problem doesn't matter the problem existed and they were near it and it wasn't fixed. If I start having those problems I want people who aggressively deal with technical problems in my technical positions.

I can of course be persuaded to hire such a person if they explain what they did and they seem highly competent. Perhaps, they did do all the right things, did go above and beyond and the culture was still so toxic as to prevent progress, I have been there before. But the interviewer would need to convince me and I think that is the point of an interview.

Thats actually a really interesting way of interviewing, and probably matches the way most people have to work.

I think the best part of this is how many people are getting it wrong in the comments here or not fully thinking it through. Perhaps it's not as easy a question as you think it is in the heat of the moment during a stressful interview?

Sounds like an effective filter to me. Where I work, this would be the first part of one of the interviews, with the second part being something more like "Given a list of calendar appointments, what is the maximum number which can be attended without conflicts?"

don't you need a little bit more info about that list of calendar appointments? like at least an assumed time length?

(assuming the list happens on the same day)?

Yeah I was thinking each appointment had a start time and end time.

> not fully thinking it through

What I like about pklausler's example is that it allows for people to find edge cases without too much technical knowledge. If I have a candidate who doesn't ask about the inputs (e.g. "are the appointment sorted?"), I'd be wary. Engineers are often given vague specifications they need to clarify or account for.

I don't think it's a bad question to ask... just not the filter that the op thinks it is. all these people who are sitting in the comfort of not being in an interview and they are still getting it wrong. this filter should be easy enough to get the question right if you know any programming at all. something like the fizzbuzz question.

I highly doubt I would ask if the inputs are sorted. I wouldn't expect them to be (why would they be, after all?), and I'd probably just give a solution that works regardless.

I think you just received some good interview advice, unintentionally.

Not for interviewing for a company with reasonable interview practices.

I agree. Also, since we're talking about two entries here, there really isn't much work to do...

One could even argue that relying on the order of the items in this case is going to result in a worse design overall (it's one additional thing the caller can get wrong).

If you want to test for the ability to gather requirements and clarifications from vague requests, then test for it directly, don't bake it in as a hidden test you expect people to account for when the overt problem is just implementing a comparatively crystal clear toy question even if you say things like "number" instead of "positive integer". Assuming they can implement anything that seems to work at all (which is your main goal, right? or why ask it?) you could turn your prior vagueness into an opportunity to see if they can formulate test cases and refactor their approach to account for different inputs, if needed, or if you see them accounting for them as they go you can ask e.g. "why are you taking the absolute value?" and see if they're thinking about the input order or if they're just doing it because that's what they saw somewhere.

Yeah, I think the important thing is that the candidate should not have to "read the interviewer's mind" when it comes to what style of question is being asked.

Unfortunately, this can also occur because the interviewer isn't clear on their own goals in the first place. They asked the question before really knowing what data points they want to come away with.

Perhaps it's not as easy a question as you think it is in the heat of the moment during a stressful interview?

This point cannot be underemphasized. "Thinking aloud", not just in front of another person but under a very stressful situation (for most people) that also basically almost never occurs otherwise in daily life -- is a specific metacognitive skill that for many people can only be (even adequately) learned by either (1) careful training or (2) surviving many repeated failures, and finally getting at least partially acclimated to these kinds of confrontations.

Yeah sure, perhaps some of the people who "bombed" that calendar test really were the walking fraudsters the interviewer (who knows they'll get to stay in their well-paying job regardless of the outcome of said interview) makes them out to be. But most are probably simply nervous, and due to a variety of psychological factors (imposter syndrome, among others) are simply momentarily blanking out, and having suffering from a very common form of mild anxiety attack which makes the problem seem much more complex to them (at that moment) than it otherwise would, under more natural circumstances.

Being a professional programmer is not an entirely stress-free job. Often, time matters and you have to think on your feet. That's why I think rejecting a programmer for losing their core competencies like problem solving, i.e. freezing, under a medium amount of pressure is valid.

I've been a developer for years and worked for multiple companies: the most stressful professional situations I've ever have been in have been maybe a 4 out of 10 stress-wise, whereas interviews range from 7 to 9 out of 10.

Interviews aren't ordinary job situations: they are contrived situations where people are making life-changing decisions based on a very short interaction.

Same here, and have worked on some extremely high-profile, complex things. Never _ever_ have I been as stressed as when asked to perform interview whiteboard questions. It simply doesn't compare.

Sometimes, you just have to appreciate the Torvalds-level of arrogance expressed in some of replies on HN.

You're reading too much into it. That wasn't Torvaldsian at all, by my reading.

The most stressful professional situations I've ever have been in have been maybe a 4 out of 10 stress-wise, whereas interviews range from 7 to 9 out of 10.

Agree mostly -- just that I guess I've fallen into more abusive environments than I should have allowed myself to over the years, where the stress levels have hit will into the 7-10 range.

But in healthy environments -- even when I really did screw something up (sometimes quite significantly) -- it's never above the 4-5 range.

Interviews aren't ordinary job situations: they are contrived situations where people are making life-changing decisions based on a very short interaction.

And pretending that they can read the tea leaves, and make that kind of assessment as to "whether this guy can handle a medium amount of pressure", to boot.

It really depends on the job. stress in general is one thing but stress on a pedestal is another. some types of people(programmers fall into this category often) are not good at embarrassment so they shut down and can't think. it's not bad because they can be brilliant without the social pressure but with that they are not ideal. if the job doesn't have that requirement it's silly to drop them for it.

"time matters and you have to think on your feet"

I've been in many stressful situations at work - none of them involved finding algorithmic solutions.

I once spent 2 days troubleshooting a machine vision system at bolt factory. Physically in the factory. On my feet. Standing behind an active vibratory hopper filled with bolts, while stake-holders were losing money hand-over-fist, and angry machinists who looked like a motorcycle gang stood next to me. And it was a Windows NT machine.

Not quite all about algorithms, but that was part of it :-)

Yes, and I've set up a demo of an interactive TV system in a cupboard at a large London hotel (standing room only!)...

Also found that the probability of someone finding the porn in the demo by randomly pressing remote buttons pretty much approaches 1 as number of button presses approaches infinity...

I once had to diagnose and fix a bug in a financial system that went belly up under extremely heavy load -- causing the company lose tens millions of $'s per hour (literally). Turned out to be a timing problem caused by satellite comms latency. The solution required us to modify how a couple different algorithmic worked. That situation was highly time sensitive and over-the-top stressful (for me anyway).

I'm sure you would acknowledge that's a very exceptional situation, right? That's the point of this whole subthread: that this kind of stress is exceptional in a work situation.

I would agree that it was exceptional. But I would also argue that how your people perform during exceptional situations is what makes or breaks teams and companies.

No one doubts that performing under pressure is important.

It's the idea that this kind of pressure can in any meaningful way be simulated (and the candidate's response adequately gauged) using the shticks and routines that people try to do in the current standard interview process that people take issue with.

This is far too much of a simplification of (grand)Parent.

you don't know the circumstances that 30 year old programmer with a 2 year old child at home is facing. You could also be like certain firms and immediately nix him on the "merits" of ageism.

A female programmer might have travelled 2000 miles to just receive an abstraction question out of left-field over an irrelevant concept to the job. She flew out on this uncertainty, and she's in a rather discriminatory field.

26 year old dev has probably built CRUD apps all his short career. and, you're asking him to implement (isBinarySearchTree Boolean) on a whiteboard on the spot.

> 30 year old programmer with a 2 year old child at home ... nix him on the "merits" of ageism

A choice, conscious or not, to dedicate time to raising a child at the opportunity cost of myriad other ways to spend time is not related to ageism at all.

Most programming jobs do not actually demand one make this form of Sophie's Choice. Perhaps the Valley is so diseased, but it's not the only game in town.

> Perhaps the Valley is so diseased, but it's not the only game in town.

Hear! Hear!

One of the things that bothers me a lot about HN is the near assumption that if you're a software engineer and not in SV (or have never been in an SE job in SV) - you don't count as much.

I'm biased, I will admit: I've never held a software engineering job in the Valley. That opportunity has never occurred, nor have I tried to pursue it. I live and work in Phoenix, Arizona; at my age and position in life, even if the opportunity did present itself, I'd have to think long and hard about it. It would likely be a situation of me leaving my family to go work there, as relocation would likely not be a realistic option unless the pay was over the top (I am not going to move back to an apartment; my next house better be at least as large as my current one, and include a larger yard and/or shop space before anything else).

Here, in my current position, I am not only the oldest developer on the entire team (and I am only a few years younger than the owner of the company), but I also am one of the few who doesn't have any children. My wife and I made the conscious choice not to have kids a long time ago, and we are constantly glad for it. In fact, she thanks me on a monthly basis for not "knocking her up".

Honestly, we'd probably make terrible parents, and we're pretty selfish as a couple. We know this, so why bring children into the mix, right?

That said - here in Phoenix (where it's actually difficult to find competent software engineers for any position), I've never seen or experienced there being an issue if you have kids; every employer I've worked for has been extremely flexible.

In fact, both at my current employer, and my previous one, they encouraged you to leave when your day was up. If the end of your day was at 5pm, they didn't want you stay a minute more if there wasn't a really good reason for it. Like things would have to be well and good on fire for that to happen; anything less was "go home, get some rest - it'll be here to fix tomorrow".

And now imagine you have 4 little kids like me and how that affects the equation! You want me to move out to Seattle, Amazon? Do you have any idea how much you'd have to pay me to have anything like the lifestyle we have in the Chicago suburbs?!

The attitude out west is "You want to earn enough to raise a family? You should have thought about that before you had kids".

It also seems to me that the people who have the most trouble with work/life balance in development jobs, at least out here, is that they are too afraid of saying "no".

> the people who have the most trouble with work/life balance in development jobs, at least out here, is that they are too afraid of saying "no"

absolutely true (at least in my case). I'd rather do that than end up on the other end, though. Without stronger protections for employees, I'll hold out for a stronger financial position on my own before I start negotiating hard about "work/life balance".

What I've noticed often with coworkers, and even myself at times, is that management often won't do anything to discourage your workaholic tendencies - but it was never actually an expectation in the first place - just a belief in the employee's head that it's what's required of them.

When I've been through such phases, once I realized that I could set reasonable expectations, it was never actually a big deal. Eventually I even started taking my vacation days, nervous that my job wouldn't be waiting for me when I got back, but that was really all in my head.

The kinds of pressure are different - unless if at your organization you do your time-critical coding in front of a panel with varying mixes of what looks like disappointment, boredom, impatience and eagerness to "give you a hint" on how they would solve it while thoroughly confusing you. If not, you are measuring the wrong metric to gauge potential candidate success.

Such kind of situations are very common where you are relatively new, handling software for a safety critical system and entire production line is shut down due to some exception occurring in large complex hard-real time multithreaded s/w module developed by some past ghost employee who has forgot to comment the code with no documentation available and the plant manager is banging your head for downtime loss!

I don't put out fires under the watchful eye of a would be colleague or employer.

Some developers are expected to work on the systems that processes a bajillion dollars of whatever per day and when it screws up they are often on the line for it.

I am not saying it is right, I am just saying at some shitty places it is.

Right or wrong, in some systems when something goes wrong, somebody dies. At some point, there's somebody on the line for that, whose responsibility it is to say "This code is good enough".

That's a high-pressure situation, so if you're hiring for that, it might make sense to interview under pressure.

> Right or wrong, in some systems when something goes wrong, somebody dies.

I have worked on systems that could cause operator death or disfigurement or damage... they are tested quite thoroughly in general and have reasonable failsafes and redundancy. It's not like they are coded on a whiteboard.

being on site at a customer facility while there is a ton of noise does stress you out a bit but most of the coding and testing is done in the office.

> That's a high-pressure situation, so if you're hiring for that, it might make sense to interview under pressure.

I think the jobs that have that kind of pressure are rare and interviewing like that should be the exception not the norm.

The stressful situations tend to be things like "We just broke the production database, we need to figure out how to recover the data ASAP"... not "We need a new algorithm in FIVE MINUTES!"

The interview question I was referring to is the appointment overlapping one. It's a middle school math question and it's a lot easier than failing over a database without data loss.

I've been doing this for 20 years and I'm not even sure where I'd start on your middle school math question. I hate working with calendars and dates and I don't think I know anyone who enjoys it and can write totally bug free code that takes care of all edge cases.

I'm sure good at fighting database fires though.

This probably is at the heart of the answer to the question "how could this programmer with such a long resume have botched this interview so badly?" It would be insane to believe they've never added value to any organization and have just been coasting along all this time. Maybe your interview sucks and isn't good at finding value.

Don't over complicate it. Given two number ranges, of UTC timestamps, determine if they overlap. I think you know where to start with that.

I'd probably go for 4 if() checks, some 'less than'/'greater than'/'&&' checks should do it.

>We just broke the production database, we need to figure out how to recover the data ASAP

I know you did not mean it this way, but in some ways that's even worse. If that type of thing were routine enough to merit asking every interviewee that question, I would seriously reconsider working there.

>Being a professional programmer is not an entirely stress-free job. Often, time matters and you have to think on your feet

Not once in my career had I had to fix a problem in an hour, let alone 10 minutes.

I have had senior people get mad at me when I couldn't give an answer during a meeting - that's the closest to an interview type scenario where you need the answer now. But even then, I did not stress that I'd lose my job, which is similar to the stress the candidate is facing.

Do you work on services that are expected to be working 24/7? It's not exactly uncommon for a production service to have problems and need to be fixed ASAP.

Do most programming jobs expect that? I think not.

It's like interviewing a doctor who will be someone's primary care physician. Give him/her a set of symptoms, and fail him if he doesn't diagnose it inside of 10 minutes. And then justify it with the expectations in surgery or ER.

It's not exactly uncommon for a production service to have problems and need to be fixed ASAP.

Yeah, but if so it's usually of the form of "oh shit, forgot to chmod this little bash wrapper".

To the extent that it's of the form of "Solve this cute little dynamic programming problem I heard about from some company where everyone is like, gosh, oh so smart! In like, the next 5 minutes, k?" -- well, basically never.

Why not go all the way, then?

Armies train people by having them run obstacle courses with guns being fired all around them. Do a "realistic" interview, then, to really test how someone does "under pressure". Have them sitting alone in the room, and a screaming angry manager runs in yelling obscenities at them about how the production system is down, it's costing the company millions of dollars, and if they want to have a job they'd better get it fixed ASAP!

And have that same manager stand over their shoulder the whole time, yelling yet more obscenities. If the candidate doesn't get it fixed in, say, five minutes, then obviously they lack even the most basic qualifications to work as a programmer and you can reject them. Heck, save money and don't even do it on-site -- get their phone number and do it 3AM on a weekend, just like a real on-call situation!

Or... maybe the whole "under pressure" thing is just an excuse people use to cover for the fact that they like making people squirm, enjoy the feeling of power they have from knowing someone else's future is in their hands and that person knows it, and want to savor by making the trained monkey stand up and dance on command... or else.

Unfortunately, people who like the "under pressure" idea of interviewing seldom realize that sooner or later they're going to be the monkey and someone else will be the organ grinder.

Relevant comedy skit: https://youtube.com/watch?v=iRtBvo9grLw

Being a professional programmer is not an entirely stress-free job.

Sure -- it's just the tenor (and for some people, sheer intensity) of the psychological stress experienced during interviews is, for many people, basically orthogonal to the stress of real-life work situations -- or even genuinely dangerous (even physically threatening) situations in life, otherwise.

With the latter, it's like "Oh shit - the client's gonna get real pissed if I don't figure something out real quick".† Or even: "That guy looks like he's about to lunge at me - I better think of something!" For which your brain and your glands have benefitted from millions of years of evolution in support of mechanisms for pumping just the right kind and amount of juice into your system to "figure something out".

But with interviews it's more like: "This person's evaluating me, using some hidden criteria. Given that the problem is slightly tricky, I literally don't know if they're expecting to power on at all costs, or, secretly, that I simply admit that I don't know where to start just yet. Not only that, something tells me he's not stating the problem quite correctly -- they do that, no all the time, but way too often in these kinds of interviews. And on top of that he's just being plain overbearing... and now he's starting to fiddle with his phone. On top of starting 15 minutes late. Like he never really wanted to talk to me in the first place. If I ask too many questions -- or even just one instance of the wrong kind of question -- I'm screwed, and gonna have to hit my inbox for leads again. Oh great, now he's starting to offer 'hints', completely derailing my flow of thought and whatever self confidence I thought I had. Fuck it -- I just should bail and go work on my personal projects. No one will be paying me $150k but at least I'll be learning something."

† Where "the client is gonna get real pissed" is roughly equivalent to "if I can't get these rocks to flake in just the right way, we aren't gonna have any more arrowheads and were' all gonna starve!", to a first order approximation.

> Being a professional programmer is not an entirely stress-free job.

understatement of the year.

This is why it's important to start the interview with culture/work history questions to probe how they're feeling, watching their body language, before moving on to simple questions of increasing complexity.

I've literally started nervous people with "naively print the string 'hello'"... After warming up, the best one eventually completed my max-difficulty questions.

This is the way to go.

I would add that it takes significant experience and preparation to interview people effectively. This is a difficult-to-learn soft-skill that not everyone has.

Too often folks just get dragged into an interviewer situations with no planning or coordination.

I wonder how all those people unable to answer interview question passed school.

I don't think they are fraudsters, they are simply not too good programmers who can't solve simple problems by themselves. Many dudes believe themselves to have technical talent just for memorising done basis. That does bit imply problem solving ability.

Momentary blackouts happen - rarely.

I've had it happen - just in my past job search the past few weeks, I had a total blankout in my last Google interview session in person on a dynamic programming question, and I'm pretty sure that ended up being the reason why I got rejected as I felt pretty good about all 4 of my prior sessions.

I am about to start work as a senior engineer for Apple with 4 1/2 years of experience, after passing two back-to-back onsite interviews with them, and I hardly have studied specifically for technical interviews. I possess an MS in mathematics.

Don't presume anything about an individual just because the person happened to not answer a particular question - the person could be actually much smarter than you. It could have just been a bad day, or a number of factors.

I know it happens, but I don't think it happens to majority of people. Bad days are caled that way because they happen once in a while, not constantly. You had good three interviews and one bad at Google. You passed well at apple. That really sounds like bad day.

But then there are people for who practicaly any question beyond "lets talk in general about what you like and how passionate for technology you are" is unfair and causes black out entry single time. Thay is entirely different.

I disagree - different people could have dufferent triggers for a fight or flight response. For one of my best friends, anxiety is her trigger.

It is super important to try to ease a candidate into being comfortable to get the important information you need as an interviewer, as interviews are inherently stressful for candidates, and you almost never want to be testing a candidate's stress coping abilities.

I agree, but here the topic is simple question being considered unfair, because someone might be too stressed. I would argue that asking questions is not unfair level of stressing people.

It's not simply a matter of "asking questions". It's about getting asked (often gratuitously difficult)† in an intrinsically awkward and confrontational situation -- quite often by someone who not only might not have adequate social skills for the task; there's a good change they may not really know why they're asking that question of you in the first place.

"I dunno - they put me through this stuff, or some variant, on my way in. So now I guess it's my turn to dish it out on the people trying to climb in after me" seems to be about the order of reasoning in place behind these "methods".

† As in, was an open problem in the literature and/or folklore for years before some (now) highly respected came up with the first reasonably adequate (but like as not, overcomplicated and/out outright flawed, at the outset) solution. We know the "usual suspects" that get asked -- I don't need to list them here. But ff you genuinely think that any candidate you can (reasonably) expect to find in response to your muddled job postings will genuinely be able to solve these problems for you at the whiteboard for you -- from the void, as it were -- as opposed to regurgitate carefully practiced solutions they found from cobbling together lists of "problems Google and FB like to ask" -- you're seriously kidding yourself.

This thread is about calendar question - figure out whether two appointments overlap. That was not open problem for years and you should not need to memorize solution to solve it.

There is nothing to suggest that parent poster has bad social skills or that his job posting was muddled.

> but I don't think it happens to majority of people

Do you know "majority of people" or have interviewed them?

Have you yourself faced an interview question at an abstraction level far below your area in this field?

Observation after seeing countless schoolmates answer questions for grade. There were only few people that would be blacked out regularly no matter what age.

Yes, I have faced questions I could not answer. I did my best and that was it. That is however not blackout - that is me not knowing the answer. Blackout is when you temporary can't recall.

But that's the thing -- it literally takes just one person out of 5 or 6 to say "I don't know about this guy" to get the rest of the team to pass. No matter if he didn't seem all that well prepared or interested in you, you guys just didn't click, or you were exhausted because no one thought to let you get coffee or use the restroom.

When people are actively watching me to judge me, my brain just goes blank. It's happened to me almost every time in technical interviews where I was asked to solve a problem in front of the interviewers, and it's happened to me at school too in oral examinations. Written exams were not a problem.

This never happens in normal work, even when working under pressure with a deadline that has to be met and not enough time to make it.

It's just a completely different setting and purpose. In one case I'm solving a problem for the sake of being judged and that judgement may impact my whole life for the coming years. In the other case I'm solving a problem because there is a practical need for it and I'm just doing my job.

It has nothing to do with being a good programmer or not, it has nothing to do with my actual ability to solve a problem either.

Suffering from the same problem, though it happens to me even in written exams - doing something for the sole purpose of being judged makes me incredibly uncomfortable, and I'm happy to be out of school/university and just have a job. It's all just infinitely easier and less stressful, even at the worst of times and with the worst of people.

Momentary blackouts happen - rarely.

Funny thing -- these folks have, y'know, actual data on this stuff:

http://blog.interviewing.io/you-cant-fix-diversity-in-tech-w...

Choice quote:

As you can see, roughly 25% of interviewees are consistent in their performance, but the rest are all over the place. And over a third of people with a high mean (>=3) technical performance bombed at least one interview.

>> "Thinking aloud" ... is a specific metacognitive skill

Is there evidence for this claim? I'm inclined to believe it but that's experiential. It's also something I've never had to work to acquire, but that could be a cultural thing (my family and friends talk a lot about thinking) or a personal history thing (I have a minor learning disability, and it has forced me to develop strong, conscious meta-cognition).

It's not a given, tho, that being able to talk about thinking is distinct from being able to think and being able to talk.

I think psychology has a lot of models of cognition without a clear winner, so there's probably no conclusive answer at the moment.

But talking through a problem forces you to use verbal reasoning. A lot of programming can be done well with non-verbal reasoning skills.

Personally, quickly sketching some timelines on a piece of paper would be the fastest way to get the correct checks. Talking it through would force me to convert my mental visualizations into words, and that'd make me stumble.

the idea to "think aloud" is downright silly.

people who "talk to themselves" are labelled idiots. mental dialogue is supposed to be mental.

call it social conditioning if you will. spelling out your thoughts for someone else to hear them just so they can take note of them for evaluation is the polar opposite of normal human interaction.

Let's just say that it's at best a corner case in the the toolbox of approaches our brain has for tackling problems. Yes, it's something we make use of once in a while. But more like a "caught you in the hall, here's my idea" approach. Never in a "solve this now in 5 minutes or yer OUT!" sense.

The problem with the modern interview process is that not only elevates the later approach way out of proportion to its actual significance -- but basically makes in the centerpiece of your interactions with the company and potential team members, on that fateful day.

This point cannot be underemphasized.

Edit: that should be 'overemphasized', of course.

I agree. Most companies screw themselves with the technical interview by giving it. I give something simple like FizzBuzz and that's it. My big questions is: "What was one of your favorite projects and what did you do?" That will tell you everything I need to know.

The problem with giving technical interviews is you are testing for someone who is an extrovert that can bullshit under pressure. That's not what you want. You want someone who is smart and can solve problems with a compiler.

I think you have it the other way round.

Asking about favorite projects is easier for extroverts that can bullshit under pressure than technical interviews.

It's also much easier for introverts. They can talk about something that is not themselves and something they know very well. They can now expose many interesting and subtle details of their craft.

Do you think introverts don't like to talk and to brag?

Yes, this. Most introverts I've known will talk endlessly about stuff they like. Ask an introverted Star Wars geek about Star Wars. It's the meaningless smalltalk they have trouble with.

Ask an extrovert bullshitter who couldn't care less about Star Wars about Star Wars and you can easily tell the difference. (as long as you yourself aren't an extrovert bullshitter who couldn't care less about Star Wars)

Bad idea.

I have not interviewed many people yet, but already during one lunch (!) interview (where I don't ask any questions), a guy was so proudly talking about his achievements at his previous job in quite generic terms, that I've got an impression he was teached the whole story, rather than actually lived it.

Later he was not hired. According to (trusted) others in the interview loop he did not know how to do any basic shit.

But you could tell something was fishy by listening to him. You said you got the impression he taught the whole story rather than lived it. That's exactly what you are looking for; that's why it's so valuable. A real developer can tell when someone is bullshitting him in about 3 sentences.

Having said that, let me be clear. This isn't a good tool if all you have to do interviews is a manager. They probably won't be able to tell the difference, especially if they are just generic managers with little to no technical experience. They aren't the type you want giving technical interviews.

See "Embarrassing code I wrote under stress at a job interview" which was discussed on Hacker News here:

https://news.ycombinator.com/item?id=9478906

I have similarly asked "write a function, minimum(), that takes a list of integers and returns the smallest integer in the given list" for quite some time. It's disturbingly effective.

I've always asked something that involved a loop, to ensure that candidates understood how to write a for loop.

(I have a variant of the above, a "more complicated" question, that involves maintaining two pointers/iterators; that removes another huge chuck of candidates…)

Funny, I remember being blown away when somebody asked me this question on my first internship interview. Only later I read about FizzBuzz and became enlightened ;)

A decently clever candidate could sneak out of that first one without a loop. Something like

    numbers.reduce((a,b) => Math.min(a,b))

would do it in JavaScript. In general, if a candidate realizes they can use reduce for that purpose, in my experience interviewing candidates they probably wouldn't have much trouble doing it with a loop, and it would usually be a signal they're going to have a pretty good interview.

Then again, you might run into a joker who has just memorized how to use reduce for finding the maximum or minimum element in a list of numbers.

I find that's a telling way to determine the experience of the interviewer. I'm saddened by the number of times an interviewer has asked me, "What is a reduce? How does it work?" Of course, that information isn't very helpful in the flow of an interview, it's mostly just upsetting.

As of yet, nobody has used reduce(). If they did, and especially if they followed it with a coherent explanation of what reduce does & how it worked (and convinced me they've not memorized reduce for a singular purpose, but understand it well enough to apply it generally), I'd definitely accept it.

If they can also write, or at least quickly approximate, the for-loop solution too, then I'd say I've gotten an excellent signal: not only can the candidate write a for loop, but he or she might also have some experience with a functional paradigm from somewhere.

The whole for loop thing - absolutely!

We used to ask people to right a function in C that converts an array of doubles from degrees to radians. It was amazing how many people had no idea and who had sailed through CV/resume screening, telephone interview, and 'technical discussion' part of the interview with ease.

If those people really "sailed" through your phone interview and technical discussion there is something very wrong with your phone interview format and technical discussion methods.

That's a good point, and I don't really understand what we were doing wrong.

I guess the candidates could 'talk the talk' when we were asking them questions on their CV, or general high-level questions on, say, threading or design patterns; but when it came to the crunch they couldn't write code.

Maybe the candidates were just good at generating CRUD applications and binding data to forms, and have rarely needed to process data. Not denigrating that aspect of software developement at all - it's just not what we need in our department.

This is a trigonometry question, not a C or programming question.

A weeder question I've used:

How many operations can a modern CPU perform per second: A) Thousands, B) Millions, C) Billions

We'll accept C, and B with explanation. I'd say roughly 75% of the people I've asked (who have gotten through a phone interview) cannot answer it with any ability.

People whine about the difficulty of interviews, but honestly, almost everyone we've ever hired have said the interview was pretty straight forward, and nervousness is the biggest issue. Sucky people whine about reversing a linked list or finding a cycle or whatever.

Used that on college graduates. I was surprised how many couldn't answer or reason through it. Even the ones from top CS schools.

I also realized how different schools emphasized computer architecture differently for CS graduates. I remember having to design a simplified PDP-11 in Computer Architecture. We had to take assembly language and such. But that was one specific college and I shouldn't assume all of them do that.

Eventually I moved away from such questions and took a more collaborative approach such as "let's solve a problem together". So I'd stand beside them at a whiteboard and we think through problems. I think that puts candidates at ease and it reveals more of the skill and personality traits that were relevant for us.

When the hell would you ever need to know that? Do you ask carpenters how many times their screwdrivers spin per minute? It just seems like useless trivia.

Well does it take about a second or about an hour to make one turn of the screw? If a carpenter doesn't know the answer to that in his bones, he won't be fastening many things together.

It takes significantly less than a second to turn a screw once. A couple orders of magnitude less actually, to give it some relation to the computer question.

Which really just demonstrates the point, I think. At some point things are "fast enough" that it just doesn't matter. We've reached that point with computers. Unless you are working in a niche field that needs serious compute, the sources of performance problems are almost never going to arise from issues like trying to execute too many add instructions in a given period. The delays will come from things that are significantly harder to see - network, database, or program architecture + runtime.

> At some point things are "fast enough" that it just doesn't matter.

...and 640K will be enough for everyone!

The carpenter may simply know that turning the screw is imperceptibly instant in his experience, and that if the bookshelf isn't getting built quickly, the bottleneck is virtually never because he's waiting on screws to be turned. He's going to have to make a trip to the store anyway so he isn't concerned with whether turning the screw requires a tenth or a thousandth of a second.

He's more focused on the design and craftsmanship of the bookshelf, because he's built many excellent bookshelves to industry standard effectiveness in the past and measuring the number of screws he can turn in a second hasn't been relevant to the work.

If he is good, he can reason about it on the spot. If he can't, I very much doubt that he can design anything excellent. That requires critical thinking to understand what it is being used for, what is good, and importantly what solutions are unsuitable for the problem he is trying to solve.

Sure, although I was addressing the parent comment, and more specifically, "knows it in his bones."

Whether this is a good interview question I don't know, but:

> When the hell would you ever need to know that?

I find this such a strange perspective. You're writing code for a computer. Sometimes you need to estimate how quickly it should run, or else estimate how quickly it could run with optimal code. Surely it's obvious that knowing how fast your computer does stuff, at least within 3 orders of magnitude, is a necessary ingredient to make that estimate.

And yet in your mind that fundamental fact is "useless trivia." Very odd.

I agree that it is "useless trivia."

The reason is that things today are "fast enough." These days most slowdowns aren't the result of the CPU not executing instructions fast enough. Other factors dominate, such as memory access patterns, network delays, and interfacing with other complex software such as databases.

Unless you are doing compute-heavy code, the speed of the CPU isn't much of a factor in estimating how fast the program will run.

I'm not infrequently surprised how people don't spot that something is orders of magnitude slower than it should be or pick the wrong architecture because they can't or won't do simple mental math to work out very roughly how long it should take to move some data around in memory, ssd or over the network or perform some simple computation on it.

I'm having trouble believing that people who think a CPU can do thousands of instructions per second will do well at this (or reasoning about the memory hierarchy).

I don't think there's much predictive power there. The people who are answering thousands clearly just haven't though about it before and are on the spot. Humans are just terrible with big numbers, and thousands sounds like a lot already.

You may as well ask any other sort of technical trivia question and figure the people that happen to carry around more random facts about tech are more likely to understand the bigger things that do matter. It isn't necessarily wrong it's a pretty obtuse way to make a judgment. Why not just ask them about the memory hierarchy or network delays or whatever directly?

Unless you are writing assembler, speed of your code is very distant of instructions. You don't count instructions when estimating speed. Not in higher level languages, not when working with database and definitely not in something like javascript that runs in browser.

I see your point, but hardly anyone counts "operations" anymore. Besides, what even is an "operation" on a modern CPU? You could say, an operation is an entire CPU instruction, but the truth is that the instructions you see in assembly files are not really that atomic. So then do you count each microcode instruction as an "operation"? But even then the question is impossible to answer, because depending on the type of workload your execution time will vary significantly. For example, straight up doing math with some registers will always be faster than reading/writing memory, and reading memory in a cache-friendly way will be faster than jumping all over the place, etc.

Anyway, my answer to the question would be, "a lot". You need to know the specifics of workload and cpu to be more precise.

What is the clock speed of your iPhone or android phone?

I personally have no idea and it's the one computer I use most frequently.

My phone, and your phone too (possibly) is something like 1.2 GHz per core, with 4 cores; most phones have a similar spec (and those numbers are going up, too); basically figure 1-2 GHz per core, with 2-4 cores generally.

That said, when I started using computers, my first machine ran at 897 Khz on an 8-bit CPU, and had 16k of RAM. I was 11 years old, and this was a pretty standard machine for the time, unless you had real money (and even there, on the top end - not counting minicomputers or mainframes - you'd be lucky to break 8 MHz and a meg of RAM).

But I know I am of a different era. And honestly, I've stopped caring about CPU speeds too, because lately they don't change much (top end is about 4-5 GHz per core; servers can have around 32 cores - though that is increasing too).

What should be cared about is memory usage per process, and how the software you are using (or which multiple people are using) delegates that out. For instance, with PHP (not sure with PHP 7) you have one process per user, and depending on how much memory those processes take, will ultimately lead to an upper limit on the number of users that can be served at one time. In that case, knowing your memory usage and constraints of the server could be very important (there are a ton of other factors to consider as well, I know).

No, it's more like asking a roofer how about the area he can cover with shingles per hour. Information of vital importance.

Let's be real, if a roofer was asked anything comparable to a programming question it would be to provide an accurate approximation of the total number of roofing contracts completed globally within the last half-minute on the back of a coffee-stained napkin.

But that's not what's being asked by OP. Their question is rather simple, and software engineer should be able to answer it on autopilot, without even engaging her brain. As a hiring manager I would end the interview right there and then after the failure to answer something like this.

As an interviewee, I would roll my eyes at the ignorance of the interviewer.

This is the sort of question that reveals the ignorance of the person asking it. The correct answer is "yes". What is 'modern'? There are very modern embedded CPUs that operate in the low MHz range. How much power is being provided? A 'modern' CPU can be underclocked to ridiculous levels for power savings. What's an operation? Are we talking MOV AX, 1 or are we talking hashing a string?

It's the kind of question a dumb person asks thinking they're being quite clever.

And low Mhz range is in the millions, which they said they would accept as an answer with that explanation.

Clock rate != operations. It's like you didn't bother to read what I wrote.

Closely enough related to number of basic operations though I'd say. I'll agree with you if it really just is a multiple choice question without context, but the initial poster did not present it that way, but said that explanation/discussion matters. And then your objections are a good starting point for that.

The worst thing you realistically can do to stall your CPU is miss cache every time you read data from RAM. If you do this for every instruction (which would be quite a feat in itself), you divide your instruction throughput by about 200. But even then, and even on the shittiest modern CPU you'll be retiring millions of instructions per second.

The MSP430, released in 2010, can be clocked down to 32kHz. That's pretty much the state of the art in modern, low-power devices.

And if you know that, I'm pretty sure you'd pass that interview.

And you are one of the types of person the question is meant to weed out - people so threatened by a simple question they get defensive and hostile.

And thanks for calling me dumb!

Thanks for projecting 'threatened' onto me!

You're the type of interviewer I would weed out, so by all means keep asking the question.

You don't, but if you found that a candidate carpenter couldn't tell you whether they could fasten 10 or 10,000 or 10,000,000 screws in a one-hour period, you'd start to doubt their ability in general.

It's absolutely relevant. On more than one occations I've had performance issues land on my table with the developer that wrote the code saying "it's not that bad, it's going through 100.000 elements". They simply don't have a relation to how much a modern computer can do in a given timeframe.

Usually I can guide them to understand where the bottleneck is, but every so often I have to rewite it so performance is acceptable.

No. Here are possible carpenter equivalents, all of which I'd absolutely consider a minimum low bar...

> How many nails does it take to build a house? a) hundreds b) thousands c) millions d) billions

> How many rooms are in a house? a) tens b) hundreds c) thousands

> How thick is a house wall? a) inches b) feet c) miles

Number of nails as analogy for CPU operations? That's not hiring a programmer, that's hiring a computer.

What I mean is that -- thanks to all the programmers of yesteryear -- most programmers operate at a much higher level of abstraction when building anything that big.

When carpenter will try to use toothed disc from circular saw on the handheld angle grinder(1) instead of abrasive disc and it will disintegrate at overspeed he'll know. Maybe it will be too late but he will, for sure.

(1) because "it is faster" and "my dad always did this" and "you don't pay for my saw, so shut up"

> When carpenter will try to use toothed disc from circular saw on the handheld angle grinder(1) instead of abrasive disc and it will disintegrate at overspeed he'll know. Maybe it will be too late but he will, for sure.

They do make such blades for angle grinders; they tend to be "universal" blades to cut wood, metal, and ceramic/brick. There are also "chain" blades, which are meant for high-speed carving of wood (they look like a chainsaw chain wrapped around a circular blade).

Likely, if you really wanted to do it and had nothing else handy, a cutoff blade would probably cut wood just fine (take care to watch for too much friction, though). Alternatively, a circle of file folder paper chucked in would work as well.

That's not the correct analogy, because the carpenter uses the screw driver as a tool, not as a platform. The screwdriver is analogous to code, not the CPU.

I would absolutely ask a carpenter how quickly he can complete a project, and I would hope he has a realistic first order approximation of how fast he can complete (process) screw driving tasks.

That said, I disagree with the question being discussed here, because I'd rather just ask him about his speed and performance; I wouldn't ask him how fast he can screw things in :) (that's a minute detail compared to the actual work).

Carpentry is far different than programming. But even so, and equivalent question I would ask a carpenter I wanted to hire would be, "How much hardwood flooring would I need to cover the rooms in my house?" If they answer, "I dunno" and then stare at me blankly, I'm certainly not going to hire them.

Apparently some people take whatever warm body shows up.

> Carpentry is far different than programming. But even so, and equivalent question I would ask a carpenter I wanted to hire would be, "How much hardwood flooring would I need to cover the rooms in my house?" If they answer, "I dunno" and then stare at me blankly, I'm certainly not going to hire them.

It would only be valid not to hire them if they have already taken measurements and noted them down. If they can't do the calculation to convert their measurements to square footage and then to linear feet of board (with margins to allow for different board widths) - that'd be something different.

Even there I would argue one could still say "I don't know", simply because you haven't specified how you want the flooring oriented per room/area (and if it goes on the bias, it gets tricky quick), or if there is any kind of special patterns or inlays you want, etc.

Until you have the full specification at hand, you can't really give more than a ballpark answer, which in the case of flooring calcs could be wildly off (agile doesn't really work for real-world engineering like it can for software).

And because of that the kinds of answer we would expect from such a carpenter in that situation is, "Hmm ... how big is the room and what size of board are you using? 2 inch, 3 inch, 5 inch, or so mix? That's pretty popular these days. How much extra do you want - I usually recommend 15% so you have some in case you need to repair."

Likewise, for the original question, I'd expect some sort of discussion or nuanced answer. Something like, "Well, most modern CPUs are rated at the GHz, so on the basic level, we can say on the order of billions of operations - but we're getting a lot more cores, and given SIMD and such, you could actually go an order of magnitude or two over that, in theory. But if you are doing something that takes lots of clock cycles, it could drop down into 100s of millions, I suppose."

What you don't want to hear is a deranged ranting, like some people are doing in this very thread, or "I dunno" and a blank stare.

> Likewise, for the original question, I'd expect some sort of discussion or nuanced answer. Something like, "Well, most modern CPUs are rated at the GHz, so on the basic level, we can say on the order of billions of operations - but we're getting a lot more cores, and given SIMD and such, you could actually go an order of magnitude or two over that, in theory. But if you are doing something that takes lots of clock cycles, it could drop down into 100s of millions, I suppose."

And again, none of that knowledge is relevant or required to be very competent at web development.

The equivalent question to a web developer would be how many lines of code would it take to make a bare bones site with a calculator in it? 1,000? 10,000?, 100,000? Options A and possibly B (with explanation would be acceptable).

Your question (although you don't seem to feel this way) is the equivalent of asking a carpenter: How many Watts of power will you consume while finishing this room? A carpenter who knows that may show lots of curiosity about their tools, but there are also plenty of skilled carpenters who don't know the answer to that.

Your interview question is more asking "How similar is this person to how I approach things?" than "How skilled is this person at doing their job?"

The question may be to asses tech-savvy-ness of the candidate (i.e. if he/she knows the SI prefixes, what the range of the modern CPU frequencies is and how do those values reflect on computing performance). There is one kind of programmers that learned their trade just to make a buck and don't care much about their working tools, and there is another kind that finds interest in computers in general. Interviewers may prefer the second kind.

I feel like that's a bad question to ask. An algorithm question at least lets you attempt it and you can see if they know things. But this question isn't something that people normally talk about or think about. I think older engineers would think this is a normal question to ask since they/you have seen the evolution of computers over time, and have actually seen the number of operations changing to get into the billions.

But this kind of thing may only be mentioned in one sentence in an intro computer systems course, and most young people wouldn't care about how many operations a CPU can do.

They might not care, but in that case this question essentially tests if a person understands what 'GHz' means and can think logically, which is not that much to ask.

Why would web programmer think about processors and GHz is beyond me. There are jobs where it might matter, but for most of them it is completely irrelevant.

When youre writing a complex SPA app and your performance is dragging on that cheap Android, it's nice to be able to reason about things like clock speed, memory usage, etc.

I know we can treat a lot of the technology we use as a black box, but not knowing even the basics of that technology indicates a lack of curiosity. I'd be shocked to find a competent programmer who couldn't answer the question.

A similar question: approximately how many IPv4 addresses are there?

See also: "Latency Numbers Every Programmer Should Know"

They might have been curious about different things then you. Or more interested in knowledge that helps build stuff ( still constantly learning, but choices are different).

What's next? We shouldn't expect programmers to be aware of how characters, integers and floating point numbers work on a computer?

If they work in high level language that hides all that behind "number", who cares. How floating point numbers are represented is interesting trivia, but hardly useful in most situations. In general, I would expect people to remember things that are used often and be able to learn the rest fast (e.g. understand concepts and have enough knowledge to make further learning easy).

The rest is fluff and hidden cultural test: "Do you have the same history as I do?"

> But this kind of thing may only be mentioned in one sentence in an intro computer systems course, and most young people wouldn't care about how many operations a CPU can do.

TBH, I'm slightly disturbed by these "young people" who have completely no idea how machines actually work.

But the question is silly because there are better ways to ask people if they know in case you require that they know and you probably shouldn't be asking anything at all if you don't care if they know.

> TBH, I'm slightly disturbed by these "young people" who have completely no idea how machines actually work.

I agree with you here, but I would like to think if I were interviewing someone and they told me such, I would then ask something like "How would you go about finding this information out?", and continue down that line (I wouldn't just accept "I'd google it", I'd want something more in depth, even though that would technically be correct).

So you would be completely comfortable interviewing someone that claimed to have done 10 years of systems programming, including optimization, and yet when asked that question, just shrugged their shoulders and said, "I dunno"? Even if you asked them how fast a modern processor was and they still said, "I dunno"?

In your scenario, the question makes sense.

Personally, the question reads like "have you ever pieced-together a computer?" and one could argue that the question could weed out better candidates for certain positions. A growing number of younger engineers have only used machines that come glued-together, and never had to compare performance vs. price as they chose a CPU.`

Realistically it's a) since everyone needs to go up to the cloud and back these days.

This question is trickier than it sounds. Sure 109 clock cycles per second, but estimating for actual human-perceived ops for an algorithmic problem one should use 106. (Source: USA Computing Olympiad problems.)

Would you mind explaining in a bit more detail what you mean here? I'm not quite sure where you're getting those numbers from.

They mean 10^9 and 10^6, and that each "operation" in a complex algorithm is often something like a hash-table lookup, which may be constant time, but takes closer to 1000 clock cycles than 1.

Ahh, I got it. For whatever reason I didn't associate 109->10^9 and 106->10^6.

No worries, it took me a little while to figure out, too.

We totally would interview someone that responded as you did. Thing is, I'd say roughly 50% of the people that applied for the position would respond with something like "I dunno" or "A" and be unable to have any sort of rational discussion about it.

Technically, all three answers are correct. (A) is correct because a modern CPU CAN perform Thousands of operations per second.

Not if it is turned off.

Well...this questions seems to be OK for phone interview for entry level programmers who should know MIPS, CISC/ RISC architecture basics. I would ask such questions to fresh graduate who is supposed to be working on firmware level stuff like writing custom co-operative multitasking schedulers, OS routines, device drivers etc. Probably this type of question is not suitable for a web developers or Data analyst/ scientist

The most logical and clear answer for me (assuming I didn't already screw up by making assumptions about the data format, problem definition, etc.) is:

1. Figure out which appointment starts first

2. Check if first_appointment.end > second_appointment.start

So:

    boolean AreAppointmentsConflicting(int start1, int start2, int end1, int end2) {
      // first and second refer to the start time of the appointments.
      // The first appointment is the one with the earlier start time.
      int first_appointment_end, second_appointment_start;
      if (start1 > start2) {
          first_appointment_end = end2;
          second_appointment_start = start1;
      } else if (start1 < start2) {
          first_appointment_end = end1;
          second_appointment_start = start2;
      } else { // same start time
        return true;
      }
      return first_appointment_end > second_appointment_start;
    }

That can be shortened to:

   return (start1 > start2 && end2 > start1) || (start1 < start2 && end1 > start2) || start1 == start2;

Or shortened even further in https://news.ycombinator.com/item?id=14641485

The third answer would get massive upvotes for brevity on LeetCode. The first answer would be preferable for readability in an actual code base.

Personally I find the first version a bit hard to follow, and the second one makes my head spin.

But I can understand the objection to the third version too. I don't think the problem is that the expression is too simple - simplicity is generally a good thing! - but that it's easier to reason about non-conflicting times instead of conflicting times:

If my other meeting ends by the time this one starts, we're good.

Or if this meeting ends by the time the other one starts, we're good.

Otherwise we have a conflict.

So now if you like the step-by-step approach, we can write a function that seems pretty straightforward and easy to understand:

  boolean AreTimesCompatible( TimeRange thisRange, TimeRange thatRange ) {
      if( thatRange.end <= thisRange.start ) {
          return true;  // that one ends by the time this one starts
      }
      if( thisRange.end <= thatRange.start ) {
          return true;  // this one ends by the time that one starts
      }
      return false;  // they overlap
  }

I think it reads fine as a single expression too, given an appropriate comment explaining the idea (which you'd want anyway):

  // Time ranges are compatible if that one ends by the time this one
  // starts, or if this one ends by the time that one starts.
  // Otherwise they overlap.
  boolean AreTimesCompatible( TimeRange thisRange, TimeRange thatRange ) {
      return
          thatRange.end <= thisRange.start  ||
          thisRange.end <= thatRange.start;
  }

And here is the matching function to go with either of those:

  boolean AreTimesConflicting( TimeRange thisRange, TimeRange thatRange ) {
      return ! AreTimesCompatible( thisRange, thatRange );
  }

It's all fun and games until one of your appointmets is specified in rfc2550 format, and the meeting room is on a big ship crossing the international date line on the same day daylight savings time changes for the ship's country while a new timezone is voted into effect. Meanwhile the other meeting room is a videoconfrerence between the international space station and a ship travelling at close to the speed of light, but the real twist is the two meetings don't actually have anyone in common!

Also keep in mind Heisenberg while traveling along state lines.

Also the meetings are in different rooms with entirely different attendee lists, so whether they conflict or not is irrelevant.

Unless there is some other mutable state that they both interact with that may cause a race condition. :)

If one meeting ends at t1 and the other meeting starts at t1, do they conflict?

To which meeting does t1 belong? One, both, neither? Can we set up an event at t1 which is not in conflict of either? How about two events at t1?

Asking questions like that can, all by itself, be a correct answer to the question.

I assume that the question is meant to give insight into how they solve a problem that on the face of it looks straight forward?

It isn't though. First assume that all datetime stamps are in UTC, otherwise we have opened a can of worms that still trip experienced developers. Now draw a "timeline" and start enumerating the different cases. Then you can make a reasonable solution. But I would only expect this from experienced developers or people with a higher degree.

> I get people writing doubly-nested loops over all of the seconds in the two intervals, comparing for equality

Mother of god

The problem with interviews is that there is no time for false starts. When approaching a coding problem in real life, I pick a solution that seems to be right.

Half the time I pick a good solution and half the time I've rushed in and picked the wrong one.

If I have picked the wrong one, I know within 30 mins that it is a flawed approach, but have usually explored the problem sufficiently to pick a good solution. The interview unfortunately ends after 30 mins.

I should also add that I'm not much better at this after 13 years of professional development than I was when I left uni, when it comes to manipulating linked lists and arrays. My systems design is a lot better though.

But if you can't do the calendar question and realize within (lets be generous) 3 minutes that double nested loops are the wrong approach... I think it'd be fair to question your abilities.

Having had the chance to do some interviewing over the last few years I think the most important thing on my side is to know what I'm trying to test for with any particular line of questioning. I don't always see that level of self-awareness in fellow interviewers. (The worst is the sort who say "if they don't do X when I'm asking for Y, I'm going to score them down." Do they really care about Y then?) What exactly is gained by letting the candidate go off into the deep end for several minutes before they realize on their own, if they ever realize at all, that their initial approach was bad when it should be a simple question? If the purpose of a calendar question or fizzbuzz question (or my own initial coding question, "implement is_even") is to answer "can you code?" why does it matter if the solution isn't that great? If you have an answer to the question you can follow up with things like "can you code a better solution?" to distinguish passes, but interviewers should keep everything time boxed, there's other stuff to cover with such short time slots as 30-60 minutes. I like to test some sort of presence for knowledge about regexes and recognizing an application, so I borrowed Yegge's phone number regex question from his phone screen tips, and I'm happy enough if the person realizes that "there's some sort of pattern thing to do this" because that's a google away, happier if they can write a script that uses regexes correctly, happiest if they know or get close to the grep incantation, but if they start trying to write their own parser I'm going to stop them because that's going to take a lot longer than 10 minutes, especially if I say "oh yeah there's another number format we need to extract". But that person still might be a good hire, just not knowing about regexes at all. By keeping things time boxed I can make sure all the other areas I care about are covered and can judge if certain strengths can compensate for certain weaknesses.

Yeah, instead it's like: "Here's an interesting question that I like because I feel I already understand it well. Let's see what they do with it, and then I'll go with whatever gut feeling I end up with. This is easy, I'm a good interviewer."

Here's a tip: most live programming interview questions should be answerable within a few minutes. Any approach that you think will take a half hour is the wrong approach.

"Any approach that you think will take a half hour is the wrong approach."

Unless the interviewers haven't given much thought themselves. "Hey Joe, we have to interview a guy in 20 min. and I can't remember where are those damn interview questions. Gather up a few for me, will ya?"

One question Ive been asked I really like is: write a program that print the odd numbers between 0 to 100.

It's telling that I can't even read a problem as ostensibly simple as this without wondering if there's some deceptive special case I haven't considered.

I used to disbelieve hiring managers regularly encountered candidates who actually couldn't code, or whatever other basic technical thing was reported on their resumes. Such a possibility - even the audacity of it on a conceptual level - simply seemed beyond the pale for me. I had interviewed before and not done well, and that I could absolutely understand. But I had no way of relating to the idea of someone entering an interview without the fundamental mental schema requisite for doing the job.

Then I started interviewing people for engineering and security engineering roles. That's when I realized that unless you put a lot of effort into curating your candidate pipeline, you interview engineers who literally (without exaggeration!) cannot complete anagram toy programming questions in their "favorite languages." That was a sobering moment for me.

I'm in agreement that a lot of tech hiring is suboptimal, selects for noisy heuristics and is even (sometimes) sadistic. But I've also met engineers - typically people who have not done interviewing or hiring much - who think that reports of fizzbuzz failures have been greatly exaggerated. They have not been exaggerated.

I haven't interviewed anyone in over a year, but back when I was doing it weekly I used to mentally sigh with relief when the candidate I was talking to could actually approach technical questions, even if they ultimately didn't give winning answers. It is a sad reality indeed; the bar is low enough that you have an edge by merely engaging with the problem and reasoning about it with technical competence in an interview.

Let me ask: what company do you work for and how much effort does it put into finding, attracting and most important! retaining! people who are any good?

How much say do the people working on a team have, in who gets hired? I mean people who'll actually be working with the person day to day, not some manager.

If the answer is 'we tell recruiting agencies we're hiring for vague role X, Y and Z', then why are you surprised?

Is the work any interesting? No? Generic corporate codebase perhaps? And you're getting generic people applying? How strange :)

ps. This is not directed at you in particular. I just find it strange when people don't do an inch beyond 'we pay money', and then expect a mile beyond 'i do just enough to get paid'

Attracting more good people won't reduce the number of bad applicants (it will increase it).

Retaining staff doesn't mean that you aren't hiring at .2 headcount per annum.

Assuming that there's a basic process of CV review (by team members who would work with) followed by non technical phone interview. You will still have people who fail fizzbuzz. I'm not talking about small errors reading the problem - but the inability to even put a for loop in pseudo code onto the page.

The fact that the test is required is somewhat silly - but it's hardly limited to dull corporate roles.

I can think of half a dozen ways to attract applicants in a way that'd significantly increase the quality of the applicants while at the same time significantly decreasing the number of the applicants.

See, you are assuming a basic process of a job board and anyone applying for the position.

I am saying you are so entrenched in mediocrity that you can't fathom there are much better ways.

I will say this though - they all require a fundamental shift from 'fill seats with people for minimum amount of pay, to do uninteresting work, with least amount of complaining' to actually giving a shit about people you work with, yourself, and your life :)

> I can think of half a dozen ways to attract applicants in a way that'd significantly increase the quality of the applicants while at the same time significantly decreasing the number of the applicants.

Like what?

He would have but they were too large to fit in the margin...

Original comment poster here.

I have conducted literally hundreds of technical interviews. To me, too, it was a jaw-dropping realization that some candidates can arrive who literally have no idea how to program a computer. I don't know how they get through school (with high GPAs, even), held their last job, or impressed their phone screener. I don't even want to speculate. The sad fact is that trivial, two-minute, easy-peasy exercises can and will expose some candidates as being unable to work through them, even with the most friendly and patient coaching.

So I always ask at least one trivial question that can be solved in a minute or two with a couple of basic relations and a Boolean connective.

I experienced the exact same feelings as you. Until I started interviewing candidates in quantity, I had no clue how many candidates cannot complete FizzBuzz. The reported failure rates of this simple test are completely legitimate. In my own interviewing, it comes out at about 50/50. The 50% who do pass often make some significant conceptual mistake, for which I'm generally pretty forgiving since interviews can be a stressful experience. This question comes at the end of a hiring pipeline that does 2 phone screens before the on-site happens.

Do you actually use the real well-known FizzBuzz problem, or some other exercise that you think is comparably trivial?

Given two StartDate values (SDa, SDb) and two EndDate values (EDa, EDb) of a fully qualified date type:

No Overlap -- EDa < SDb OR EDb < SDa

Overlap -- SDb <= EDa AND SDa <= EDb

I have this written on a PostIt in my binder.

Spending a week working beside an engineer (or seeing how they complete a substantial project) almost certainly provides a better measure of their abilities than watching them solve interview problems for 1 hour

Trial employment is expensive for the company.

[...]

Trial employment (and large take-home projects) are expensive for the candidate.

I can't help but think there's way to mitigate that expense among several (dozen) companies.

Triplebyte is already interviewing for other companies, why not set a low bar and hire everyone who passes it for a week? Charge your clients accordingly. If you have 25 clients hiring for overlapping skills, have them all pay to hire a person for a week.

40 hours of of labor / 25 clients = 1.6 hrs of labor per client, or about what it costs to do a single technical interview.

Meanwhile, the interviewee gets to do a single, focused project for a week, but effectively interviews with 25 companies!

I can get on Dice right now and find over 100 openings for python/java/golang/javascript/c++ engineers, certainly all of those companies would benefit from something like this.

More power to Triplebyte if they can figure out how to take those projects and turn it into a client deliverable on top of defraying interview costs.

IMO, trial employment works because interviewer gets to spend a week with the candidate personally to get a feeling of how they approach and solve problems. Having Triplebyte do that without the interviewer being there to work with the interviewee on something, then it wouldn't be any different from an hour long interview

That's an interesting idea! The hard part would be convincing companies to trust an external trial week fully (part of what's great about a trial week is the chance to actually see how an engineer will interact with your team / your problems). But I like the idea!

It's true that a company would not be able to gain a "feel" for working with a candidate, but those feelings probably help contribute to noise as much allowing interviewers to ask their pet questions.

Surely a company could objectively learn a great deal about a candidate by looking at a week's worth of git commits and evaluating the final product for correctness/maintainability/optimization.

But still, taking TOO much of the humanity out of hiring ignores the fact that ultimately, a person has to work with other people, so you're right that it's not perfect.

A trial week seems like it would only work for more desperate candidates, ones who are currently unemployed.

"unemployed" != "desperate"

Not for the first few weeks, usually. But given enough months the desperation is really just a quirk of the implementation.

It also doesn't work for people who want to interview speculatively. Looking at what else is around once in a while is a pretty decent due-diligence that you can apply to your career. You can even be honest about this with recruiters. "I'm not sure that I want to change jobs right now, but I'm open to persuasion."

If trial weeks become commonplace, it starts looking like a form of structural lock-in.

This is why I heavily push to talk to nearly everyone on teams at the prospective company that I might be a good fit for. I spend 3-4 hours talking to various people over the course of 3-4 weeks. If the company doesn't allow me to talk to their employees, then I pass them up.

If I'm already employed, it be weird to take a week off just to work for another company. Also ramp up time takes at least a day or two. This whole process seems troublesome for both the employee and employer

When did you last switch jobs?

I recently went through a round of employment where I quit my job at the beginning.

Between updating my resume/social networks, brushing up on academic CS, finding leads, scheduling interviews and follow-up interviews and managing/negotiating offers, it was absolutely a full-time job.

I can't imagine finding a new programming job while still working at the old job.

Theres no way I would quit my current job before having another job lined up, if I didn't need to. I do agree that looking for a new opportunity is like a full time job, but when you are already employed the necessity is not as great. I can take my time over months rather than try to treat it like a 9 - 5.

That seems like an enormous risk. I've in the past spent upwards of 6+ months looking for work, and plenty more people have spent much longer. Given Bay Area expenses, if I quit my income, the first thing I'd have to do is temporarily move somewhere cheaper until I found something. Contrary to all the "shortage of engineers" stories, tech jobs do not, in fact, grow on trees.

Doesn't work when you are on a visa, unfortunately. But even if you could, I still think you wouldn't need to quit before you interview, best case may be a couple of weeks off for preparing.

I think this is a crazy expectation.

I last switched jobs while working at Apple last October. Having the BATNA of your current employment is crucial to the negotiation process and I can't imagine forgoing that- if you have no current income that's an insane bargaining handicap.

Anyone with experience absolutely is not going to go for "quit your job, and maybe we'll give you a new one after a week, maybe it won't work out and you are boned"

I had a YC company that wanted me to do a trial week. They wanted me to quit my job when I said I couldn't take off a week of vacation.

When I said no, they wanted me to come out Friday through Monday and since the whole company works on the weekend, they would get 4 days to work with me.

At that point I said I wasn't interested.

That's outrageous. Starting the death march before you even get an offer.

Wow this is next-level outrageous!

Could you share the name of the company so others might avoid them and avoid wasting their time with such nonsense?

Since it was in 2014 and I don't know if they still do this, I won't name them.

I guess the lesson learned is to ask what the interview process looks like before getting involved.

However for this company, I applied online, they sent me a InterviewStreet/HackerRank coding test, and only after I passed that did a recruiter call me, and tell me they wanted me onsite for a week.

>"However for this company, I applied online, they sent me a InterviewStreet/HackerRank coding test, and only after I passed that did a recruiter call me"

This is a huge red flag in my opinion and I would recommend people not agree to this. Why? Because it shows complete disregard for the candidate. "Pass a test and you can speak to an actual human being." Its demeaning. It also shows just how useless many recruiters have become. Part of the job of a recruiter's job and good ones do this, is to get a candidate excited about interviewing with the company and get the preliminary deal breaker questions out of the way to see if it even makes sense to proceed.

And for me that was a perfect interview. First quick question by email to see if I was interested, then some simple test to see if I can really program, then quick interview to see if I have required domain knowledge (all questions were about knowledge required to program effectively, not abstract "how to move mount fuji"). For an introvert programmer like me it's a perfect interview schedule. Also it helps when you are already working and want to check conditions in other company without risk of being thrown out from your current job.

I gave up on those the last time I wasted 4 hours of my life on one of those tests and never even got a rejection email. It wasn't a hard test and I didn't do badly.

Nowadays I just point them at my github. If a body of open source isn't enough to get an interview I'm not interested in working there.

Are you against coding tests in general (especially when you have open source work to point to) or just as a prerequisite to speaking with someone?

Just as a prerequisite.

Agreed. My experience is that recruiters are pretty flakey people(and I'm being charitable in that) - they regularly fail to keep scheduled calls, fail to follow up after asking you for some times to chat, take weeks or even months to respond resume submissions for jobs they are posting etc.

I have no problem with competency testing provided I have already spoken to someone on the team and established contact and registered interest with someone besides a recruiter.

Yep, Skyscanner's HR people keep contacting me asking me to jump through hoops for them.

Initially it was a Hackerrank challenge, now its a (100% Javascript) "Full Stack" take home exercise before you even get to speak to anyone technical.

Dude.

I just finished working for a YC company that not only demanded a two week trial period, but after doing that trial period instead kept me on a contract billed for eight hour days but then insisted I must be in the office from 9 - 6:30 (their "engineering hours"). They wanted me to work close to 50 hours a week for no health insurance while underpaying hours and forcing me to attend all company events including weekend "hackathons" (haha so fun). It was a nightmare and I'm glad to be out of it.

The irony is that it was a company that prides itself on giving people stable jobs and healthcare.

This is silly. Your BATNA can be the amount a competing company is willing to pay you in another offer.

If you lack confidence that you'll be able to find at least another two job offers in a timeframe that makes you comfortable, then I agree don't quit your job.

Anyone with experience shouldn't lack that confidence though.

I'm not wealthy enough to wait for a better offer indefinitely, and the places I was applying to are for all intents and purposes. Negotiating with only X months of runway is very different.

Your argument also assumes that I have a competing offer already when negotiating, but how did I get that competing offer? If you have a job, that solves the chicken and egg problem. If you don't, you don't have the leverage to anchor that first offer, which anchors the second, etc.

Having done both at similar skill levels, I can tell you the outcomes are much better with a job in hand.

Not only that, working for a company like Apple, you aren't allowed to write any code unless its been approved by the appropriate people...that just wouldn't work.

Why would it be a bargaining handicap unless you're running out of money (i.e. desperate)? If you're a strong enough candidate to land an offer at a top firm, it isn't worth it for them to low-ball you and risk spending more resources looking for strong candidates. Besides, if you can get an offer at Company X, you can probably get an offer at one of their competitors as well (which is your leverage).

Can probably get an offer is different from currently working at one of their competitors. It's a) a vote of confidence from another firm that this person is worth hiring b) an anchor point for your salary above where they would normally start the negotiations (yes even for "above average"). It removes all the guesswork of "can probable get an offer". You have one. Right now.

Additionally I've had negotiations stretch on for months. That isn't very comfortable with no income, and is an absolutely enourmous opportunity cost.

>Why would it be a bargaining handicap unless you're running out of money (i.e. desperate)? If you're a strong enough candidate to land an offer at a top firm

The more specialized you are the fewer jobs there are out there that match your skill set. Not every job market is as liquid as that of, say, junior java developers.

Rejecting this job might mean waiting another month or two for another to come along.

Avoiding months of unemployment also isn't just about the hit to your savings it's also about the hole on your CV.

> When did you last switch jobs? I recently went through a round of employment where I quit my job at the beginning.

The tone of this implies that lining up a new job without first quitting is outdated and negative. From the other side, quitting before having a new position can look irresponsible.

I have never quit a job without another lined up and have done quite nicely, myself. Why ever invest in trashy projects when one can get good jobs quickly and easily?

I spent a year not working and had no trouble lining up dozens of interviews when I started looking. However, it did take several months before (a) my interview game was up to par, and (b) I found a company I actually wanted to work for.

What did you do for the year off? I've always believed that tech is one of the most generous industries for this sort of "resume gap" of a year+ but if the question comes up I don't think most people would think highly of answers like "sleep" or "video games", even though more socially acceptable answers like "travel" aren't all that different... I wonder how much it varies from tech company to company. Last time I did interviewing I had an older guy ask about my GPA, and a few wondered what I was up to in 2013 when I had a year off my resume (finishing off school but that doesn't answer what the summer was spent doing, which wasn't much of note).

Generally I told people that I had done a tiny bit of contract work (true) and had been working on my own project (somewhat true, although it never got past the conceptual stage). There were a couple other things, which I won't elaborate on for anonymity's sake, but they were not programming related at all. Even on the projects, interviewers never really dug deep into them.

I suppose sleep and videogames aren't great answers. Taking time off to travel or explore some other interest is almost certainly fine in SV.

Ultimately, I think most places are so hard up to find qualified candidates that as long as you pass the interview and don't come off as a total slacker, you're probably fine. But if you're career-minded and applying to a place that has their career tracks worked out (read, not most startups), you probably want to give that impression to your future engineering manager.

I've done this for nearly every job in the past. However, since moving to the Bay Area, I would never consider quitting my job just to look for another. As one commenter below said, you put yourself at a disadvantage when you're bargaining for compensation.

I can agree that it's pretty difficult to find a new job while still working. However, most companies let you work from home nowadays. You can just interview during that time. Is it ethical? Who cares? It's business; the nature of the beast.

However, most companies let you work from home nowadays.

I don't think that's even slightly true.

I have found a new programming job while still working full time at an old job about 5-6 times now. I almost never had any time off between jobs.

    > I can't imagine finding a new programming job while still working at the old job.

Imagine having things such as: a family, a car, a mortgage, bills, etc.

Giving up a job (therefore your income) just to find another job is not an option for many.

I have all those things, two cars in fact!

At the time I was living in north east Ohio, where cost of living is fairly low. It would certainly be harder in high cost of living areas.

Experiences differ, I guess. I have twice looked for a job while I didn't have one, and I found it to be way, way less than a full-time activity. Updating my resume, posting it to a handful of job sites, monitoring a handful of job sites for new job postings, handling recruiter calls and screenings, just didn't take all that much time. It was maybe a couple of hours per day.

>"Between updating my resume/social networks, brushing up on academic CS, finding leads, scheduling interviews and follow-up interviews and managing/negotiating offers, it was absolutely a full-time job."

Can you break down how much time you spent on each of those tasks that they took up the equivalent of an entire 8 or 9 hour work day 5 days a week?

I think you might be in the minority with your outlook. Most people opt to keep their paycheck and health insurance and vacation time until they find something new. Not least those with families. Also its much easier to negotiate a higher salary using your existing salary as a starting point.

IMO, it's not the literal time taken, but the mental bandwidth needed to do these things (and do them well) that can feel "full time"

to each their own of course

Sure, here's an approximate breakdown:

Morning: 1-2 hours: Study a chapter of CLRS Algos book 1-2 hours: HackerRank 1-2 hours: Studying trivia of technology X

Afternoon was all about sales - updating social networks, filling out applications, talking to recruiters on the phone, scheduling interviews, etc. I probably applied to about 250-300 companies.

I made tweaks as I went - I found that recruiters preferred to talk in the morning, so I eventually inverted the whole thing. I started to sniff out which companies were serious about hiring and which perpetually advertise for talent

This is just a rough framework. Sometimes I'd spend the whole morning on CLRS, or a tricky Hacker Rank problem, other times I'd spend the whole day interviewing.

Before I spoke with anyone at a company, I always did research on them, the company, the company's market, and the technologies they were interviewing for.

>" I started to sniff out which companies were serious about hiring and which perpetually advertise for talent"

Would you be able to share how you sniffed them out or what techniques worked for you?

In my opinion this is one of the largest time sinks. Companies that aren't really hiring.

I see this repeatedly on the HN "Who's Hiring" thread. Companies just perpetually advertise the same roles and very often these are the same companies who seem to have no problem wasting other's people time with flakey recruiters or ghosting or just never responding to candidates. I would love to know if there is a commonality you found?

I also wish there were a better way to share experiences regarding these perpetual time wasters, something akin to hosts respond percentage on AirBnB. And I personally don't consider Glassdoor to be a very good experience or trustworthy outlet.

We often hear about the shortage of talent. And while I don't doubt there is an amount of shortage, I am often left wondering what percentage of this are companies completely broken hiring process. I also wonder whether these companies are even aware of how broken their process and their recruiters are.

> I recently went through a round of employment where I quit my job at the beginning.

From my experience of hiring people and looking for a job myself, this is the exception.

> Between updating my resume/social networks, brushing up on academic CS, finding leads, scheduling interviews and follow-up interviews and managing/negotiating offers, it was absolutely a full-time job.

That is ridiculous, you must have incredibly bad time management skills. Or do you actually think this is very common?

When I first got out of college in '08, it didn't take this kind of effort to get hired. At my previous job, it was also fairly easy - but hiring has changed quite a bit in the last 3-5 years.

I had a company demand I do two phone interviews, plus an onsite, 2 time zones away, for a $50/hr, temporary contract with no health insurance. Obviously, I politely declined to continue, but companies wouldn't be doing that if it wasn't working on some candidates.

And a breach of contract in 95+% of cases this 1 week trial only really works for new grads and any how as the USA is at will what is the issue here.

    > it be weird to take a week off just to work for another company

It's not even weird, it's just downright insane. No person should be expected to essentially work for free for a week without any guarantees they will get anything out of it.

Actually, I'd really be interested in knowing if there's some list of employers like that so someone in a position to take such an offer can rotate through good prospects like that.

> Meanwhile, the interviewee gets to do a single, focused project for a week, but effectively interviews with 25 companies!

Isn't this basically what bootcamps are about? (at least from a job seeker's perspective)

In practice, only those without jobs have the time to attend one... and if one isn't actively working in the field, then 2-3 weeks might be better than 1 week.

So I pay for 1.6 hours of this candidate, but then if I want him I have to hope either nobody else wants him or that I am paying more than the others.

Is that so different from normal interviews? At least this way you can't forget you are competing.

I find it odd that so many comments here highlight the fact that people are giving interview questions that have little to nothing to do with day-to-day employment tasks.

It seems to me like an enormous amount of time wasted for both interviewers and interviewees.

When considering someone for a development/engineering role there are a few basic questions to figure out:

1. Does this person have competence within our technology stack?

2. Can this person communicate clearly?

3. Is this person productive/can this person be productive in our environment?

4. Does this person have characteristics that will allow them to succeed in our environment?

Making people jump through hoops like some sort of dancing monkey while they are on a job hunt is needlessly cruel and is a waste of everybody's time.

If you regularly see people who can't succeed with what you consider basic engineering questions, you either have a recruiting problem or you yourself have a communication problem.

If you expect an interview candidate to expend 8 hours or more working on interview tasks, you should be presenting a unique enough opportunity to justify it.

If you expect an interview candidate to whiteboard a solution to a coding problem that they received in the room, that should be part of your everyday work experience and you should go through the process yourself in front of them so that they have an understanding of what your expectations are.

If you want to review their code and talk with them about it to determine competency, provide an example of your own code and a set of questions and answers that would be successful.

Don't put engineers through a gauntlet of challenges that you would have trouble successfully navigating under the pressure of feeding your own family. They have more than one company they are interviewing with and each company has their own focus. Put them in a position to succeed and give them the chance to do so. That's what you want out of your own people - the ability to succeed given a reasonable amount of guidance and clear parameters for success. If you can't provide the guidance or the parameters for success for an interview, how can anyone expect to succeed in your actual working environment?

The question of relevance reminds me of an anecdote. (Before that though; I tend to agree with your points.)

A distant acquaintance interviewed for a secretary position. It didn't go well, because she was asked a general knowledge question (something along the lines of: "name some of the planets in the solar system"). She was furious with how irrelevant and unfair this was. It makes me wonder though, would I have hired her? I probably wouldn't have asked that question, but still. When someone is missing some really basic knowledge - even if it's irrelevant - it's not a good sign, per se.

One of the things I like to do is ask a factual question about a technology listed on the candidate's resume that isn't required to know in order to hack something together, but is something that "you should know" if you've been working in a particular technology for a few years.

These are part of the warm-up, I'll ask one or two right at the beginning of the interview and then move to the real questions.

Examples are: "How does the autorelease pool work?" (Objective C before everyone switched to ARC.) or "How do you ensure that an object is garbage collected," (C# and Java.)

These are more about judging a candidate's real knowledge versus stated knowledge. Someone who's spent 8 years in Objective C better know how to use the autorelease pool; and someone with at least a year of experience in a garbage collected language better know how to ensure that an object is collected.

As someone who has worked in C# for about 8 years: you can't.

You can call `GC.Collect()` and hope. But far as I know the GC will do a best effort and gives no guarantees.

Of course that's besides that fact that I would need strong arguments to accept a `GC.Collect()` anywhere in the code. But to call it just to ensure an object is collected? Oh boy..

Depending on the case you could add some code to the `Dispose(bool)` method to know when your object is collected, but also in that case you risk flipping a Bozo bit.

Why? Everyone has some things that are "common knowledge" that they don't know. Trivia isn't called "trivia" because it matters.

Well, aside from categorising such basic knowledge as "trivia", it's definitely not about taking one random, useless fact, and failing the interviewee on it. Rather, it's an example of a lowest common denominator question that everyone who has primary school education should laugh at, and move along. Hitting a hole at this level does imply some major issues with the person's general knowledge. It's fair to wonder what else will be missing in that person's repertoire, and how they will cope in random situations.

Similarly, if a programmer fails the question "in the alphabet, which letter comes after A", I'd be worried, and rightfully so. I'd give them a chance to recover from that, just to make sure it wasn't a once-off, but it's not excusable.

You say we need to figure a few basic things out, how do we do if a person has basic confidence if not by asking them questions.

Please keep in mind that resumes nor any other data from the candidate can be trusted, because a huge amount of candidates lie or are mistaken about their own skill. I once interviewed a candidate who claimed to have 10 years of SQL experience but couldn't write a query on the order of "select * from table_name". Fully half of all people I have interviewed failed to convince me they could write anything more complex than "Hello World".

I am not advocating hours and hours of needless hoops, but a candidate for just a about any programming job should be able to Write a working fizzbuzz implementation. Questions on the order of fizzbuzz can and do eliminate a huge number of possible candidates.

> Please keep in mind that resumes nor any other data from the candidate can be trusted

That's not a problem that is unique to software, yet nobody asks directors to prove their budget forecasting skills during an interview, or has their tech writers go through the process of building automated indexes or varied pagination.

If you suspect someone can't write fizzbuzz, send them packing. Don't waste their time or yours. They've already instilled a lack of confidence which would mean that they would be facing adversity on day 1.

Tech writers are certainly asked to create technical documents to demonstrate their skills. Chefs might be a good model too; it absolutely the case that before hiring a chef they have to plan a menu and cook.

We aren't business people: we do actual productive things.

> Tech writers are certainly asked to create technical documents to demonstrate their skills.

In an interview? Not that I've ever seen. Reviewing a portfolio is not what we're talking about.

> Chefs might be a good model too; it absolutely the case that before hiring a chef they have to plan a menu and cook.

If a chef is expected to plan a menu and cook as part of their interview process, they are going after a very high end job. They know the kind of food that the restaurant serves. They are provided with the tools, equipment, and ample prep time to properly consider their course of action. And let's not pretend that every chef has to go through a live cooking trial for every job, or that most high end chef's would not throw a hissy-fit and walk out the door if you assumed that they would show up to an interview ready to plan a menu and cook a meal.

--- "what? You've never heard of fizzbuzz? It's a simple game that..." cue a terrible explanation of the game missing key requirements that the interviewer expects the interviewee to just know.

So now the developer not only has to come up with a set of code under pressure, they have to do so with a set of unclear requirements. And what the heck, why not take away their daily development environment to ensure that they are more uncomfortable and let's have them do it on a whiteboard so they have no hope of resolving simple syntax errors that arise because this particular developer floats between four different languages at his current job.

What you see as a game to weed out a littany of liars and cheats is seen entirely differently from a job-seekers perspective. "I have 15 years experience building complex systems and this jackass wants me to write kiddie code on a whiteboard to prove what, exactly?"

I'd rather not work for somebody who operated off of the assumption that I was a liar and a cheat upon our first meeting. The only reason people subject themselves to this kind of treatment is because they _need_ a paycheck.

I didn't say that any given person was a liar or cheat. It seems clear to me that some significant fraction of the interviewing population is and I cannot tell the difference just by looking at you. So I must do something to weed out the liars and cheats from you.

I prefer to do something objectively testable and easy enough that most programmers worth hiring should excel at it. I also do what I can to reduce stress, I explain that I am human and make mistakes too. I explain that would prefer a two way dialog to a battery of questions. I also try to explain that I think requirements always have some vagueness and sometimes need clarification.

I am curious why I should be lenient on someone unable to seek clarification on requirements? Would you want want to work with someone who couldn't seek help on requirements? I have seen that person and at the beginning of my career I was briefly that person, it is not productive and can a sink project if that person is trusted too much.

You know... I re-read my post early this morning and I thought... "that was way more hostile than necessary". But it was too late to edit. Sorry about that.

I think digging my heels in further at this point is making my opinion appear stronger than it is.

Perhaps budget forecasting is more difficult to check in an objective way. It seems silly to compare that to easily checked skills like technical writing or software development.

After my experience interviewing it seems reasonable to expect that no one can write fizzbuzz and the only way to know otherwise is to have them write it.

Why are you so opposed to the idea of meritocratic interviewing? Why not check a few things that can be easily and objectively checked? Have you ever interviewed people?

maybe instead of doing this hoopadoop all the time, its time to accept that fulltime employment is a concept that made sense when most workers were doing assembly line things.

hire specialized contractors to do the things you need done.

ive never been asked stupid fizzbuzz but have so far not been able to sufficiently fail at a job. weird.

My own 2 cents for interviewing, for engineers and most other vaguely technical positions but perhaps broadly applicable:

- Don't ask them to whiteboard, especially don't ask them to whiteboard some absurdity

- Don't put a panel of 5-10 people against a single candidate, but if you must absolutely (why?) do not do it on the first interview

- Don't ask time-wasting stupid questions about manhole covers, boiling bagels on mars, overlapping clock hands, or how many eggs fit into a phone booth

- Don't put someone into a socially awkward situation

- If you ask a knowledge question and the response is something like "I don't know, but I would google it" that should be acceptable if they are demonstrably capable of using what they find

- Do ask them to walk through their own projects, their code, their past work/contributions, etc

- Do try to effectively convey the corporate culture so the candidate can know if they are a good fit or not ahead of time

- Do try to accurately convey what their actual job duties and day-to-day work will be

Can you expand on the boiling bagels on mars question ? Havent heard of that before

In context it sounds like a flippant reference to the 'creative thinking' type questions that were all the rage a few years ago. Ask someone an oddball question with the aim of having them talk through their process of solving a question they've never heard before because you just made it up.

As long as you make it clear that the actual number they come up with is irrelevant and you just want to see how they think (and you're also open to pragmatic answers like "why the hell would I be trying to calculate the weight of an Airbus A380 when I could just look it up on wikipedia?") I don't see why it's that bad a question.

It's just a variation on the typical logic/creative thinking questions that a lot of people and companies obsess over.

"Well they're a great candidate otherwise but they totally failed the boiling water on alternate planet question, so let's keep searching" - believe it or not, that's a real mentality out there. I don't get it and never have.

Maybe its just me but I find a lot of algorithmic interview questions are just as hit and miss as these obscure questions if you need to do it in an interview.

I had a phone interview with Google and was asked to come up with a variation of a soduko solver. Couldn't get it there and then, but half an hour later walking down the road, a really elegant solution came to me.

I've taken recently a row of interviews. I've had my first whiteboard exercise and it wasn't as bad as I've expected to be. It actually proved to be very useful, as I could abstract away details that I otherwise had to deal with if I had to implement the solutions on a computer. I've also had those five interviewers on the other side of the desk! It was pretty funny for me, but I concede it may have been stressful for somebody else.

I usually ask candidates to draw and explain the system block diagram of project/ module they have worked on, on whiteboard or paper!

> background-blind interviews, looking at coding skills, not credentials or resumes.

Subtext: "Did you create a famous open source tool, write a book, have patents in your name or architect some amazing system at a big company? Doesn't matter! Our hiring process prefers code monkeys who can solve our puzzles instead of system thinkers."

> An interview can result in a bad engineer being hired and later fired (a false positive).

Subtext: "A hire can either be good or bad. It's never management's fault if it doesn't work out."

It's hard to know if a patent holder or system architect actually did the work or just took credit for it. It's easy to check if someone can print out a binary tree. Certainly a "system thinker" who can't write the simplest code and assumes it's for "monkeys" to do it is worthless both as a programmer and in any higher-level capacity, whereas someone who can program might be able to do meaningful higher-level work.

>Subtext: "A hire can either be good or bad. It's never management's fault if it doesn't work out."

I agree that the article gets this wrong. It reminds me of Diego Forlan's time at Manchester United.

Diego Forlan is/was an Uruguayan striker at Independiente (Arg) where he scored 36 goals in 77 games (a very good rate). He then moved to Manchester United (Eng) in January of 2002 and was anticipated to take off. He made 18 appearances in the 2001-2002 season but failed to score. It took him until October to net his first Premier League goal.

He was loved by the fans as he has/had a great attitude and had some delightful near misses. He also famously scored twice against Liverpool[1]. But he continued to struggle at United, scoring only 17 goals in 63 appearances until he was sold to Villareal (Esp) in August of 2004. In his first season at Villareal he won the Pichichi Trophy (golden boot for La Liga) and the European Golden Boot. What a turnaround!

Forlan went on to have a terrific career and is considered one of the best Uruguayans to play modern football (soccer). His time at United was certainly a misstep but it wasn't because he was bad. And it wasn't because United were bad. It just didn't work. Sometimes it just doesn't work.

[1] https://www.youtube.com/watch?v=49Zym0gaZqA

Delighted to see the unexpected Diego Forlan story here! Such a great talent and the thing was, EVERYONE wanted him to do well at United. I don't believe it was a case of management or team mates not liking him or not working well together, it was just some strange combination of factors that resulted in his performances never living up to expectations.

It really does draw parallels to other jobs as you say. I recall a company I joined that had a great mission, interesting work, friendly team and lovely atmosphere but something about it just didnt gel with me. I still have no idea what it was but I guess sometimes it just doesn't work.

Did they really get it wrong? Very few candidates are Diego Forlan, very few wrote books, verfy few created a foundational open source project, etc...

I suspect candidates that wrote the Linux kernel, Rails, Modern Effective C++ etc... don't need a recruiters services. If whatever candidate wrote is not so big as to land them jobs almost automatically then was it really that big as to demand consideration separate from college degrees and other ignored history?

I think we dislike the idea of our history being ignored because we all feel that we are special, but in reality very few of us are.

>Very few candidates are Diego Forlan

How would one know? Usually we know a friend who works somewhere and they say the company is shit. Or you work somewhere and someone doesn't work so you end up thinking they're shit. It's not often that you have both sides of the story.

>very few wrote books,

Books might indicate a good communicator and independent ability to complete a task but it's not an indication that someone is a good team member that fulfills a particular role.

>verfy few created a foundational open source project, etc...

I contend that this actually risks being a Balotelli. Good in the right atmosphere, but very independent and often stroppy when they don't get their way. They can be terrific in their element but often they're not good enough to build a team around.

I would argue I know because I have done many interviews, not nearly as many as the author of the article. Most interviewer candidate are about as qualified as a stuffed animal, except I would allow a stuffed animal in the office because they tend not to break things.

I agree with you that on your points about books and project founding, but I would hear a candidate out and not make hard unchangeable presumptions.

As for how do I "know" I can't know, I am only in the interview for an hour or two. The best I can do is get a sense. If the candidate nails fizzbuzz, explain the nuances of git v mercurial for 20 minutes and provides a plausible explanation for his previous place of employments toxicity then that affects my decision making one way. If it feel as though their reasons are thinly veiled excuses, they don't know foundational tools or fail at fizzbuzz... well I don't let interviews proceed to where people fail all three when they start with some minor points against them. Too many red flags and I call it done.

An interesting point my co-worker made is that, almost by definition, if an interviewee does not know how to answer a question, they don't know why that question is important or where that type of problem is applied. So from the point of view of an interviewee who's can't do well on all the questions (that's most of us in most interviews) some of the question will always seem like minutiae / lacking application. (Of course, there's also plenty of just bad interview questions)

It's entirely possible to not be able to answer a question because the question is objectively not meaningful, and therefore the candidate has correctly never considered it important enough to learn about.

Totally. Both things happen.

Sometimes questions are useful only if you are in the context of the specified problem. I mean, if you are not struggling or have never struggled in the problem, you will probably not know the specificities of it. This is specially true for unexperienced engineers or those who are working on a different field

(Of course, there's also plenty of just bad interview questions)

Well yeah, there's that unfortunate fact. The problem is, a lot of interviewers can't tell the good from the bad -- and it can only take one bad question to effectively sink the interview.

Where do you see yourself in 5 years?

Every time I hear that question, I die a little inside. I'm not sure it's possible to answer it honestly without taking yourself out of contention.

I suppose it depends on your 5-year goals and the position, but as an entry-level developer my first thought would be:

"Well as I'm interviewing here I would obviously see taking this position as a step foreword for all the reasons we've discussed. Naturally I don't see this position as the job I'll work until I retire, and obviously I'll be looking for at least some mild expansion of responsibilities in the future. Whether that happens here or somewhere else in 5 years remains to be seen, but staying here is certainly an option depending on how things are going after 2-3 years. On that note, exactly how are company name's technical and managerial tracks structured..."

In that paragraph I've been honest, I've let them know that I don't intend to do menial labor forever and have aspirations, the minimum amount of time-loyalty they can expect from me, an assurance that I won't job-hop, and I've shown further interest in their company as well as the initiative to ask questions of my own if I haven't done so earlier.

Edit: Reading it here may sound like I want to be a manager in 3 years (which is crazy for a junior dev), but that's where manner and vocal tone come in.

Ok, now do it for someone with 5 and 10 years of experience.

That question is easier with experience, because you know more about how positions look like in practice and what you disliked in the past.

Here's my honest answer:

I don't know, because based on my experience, your company won't even exist as a separate entity five years from now, and if it doesn't lay most of us off before the acquisition, the purchasing company will just afterward. So I'll probably be sitting right where we are right now, answering the same question again. But maybe--just maybe--the company will last that long, and grow, and I'll be sitting in your seat, asking it. But since I am where I am, that obviously hasn't happened yet.

Dude, you absolutely should be trying to be a manager in 3 years, if you want to go down the management track.

Depending on company size, you should probably be a team-lead or something by then.

Junior developers that remain "solo" workers after 3+ years in the industry is a bad sign for value. If you've been developing for a while, at the very least you should be put in a leadership position from a project perspective. Management/leadership experience doesn't just pop up out of nowhere. It's a skill you have to work really, really hard at. Take every leadership opportunity you can grab. You don't have to be "the boss" in the formal sense of the word, but you absolutely should be working towards a leadership role.

Depends on the industry. None of my current managers have anything less than 10 years experience, even at the lowest level (I'm at a big old defense contractor). First promotion from associate developer to staff developer typically comes after two years, and it's not due to lack of talent. Likewise our team leads and SMEs are often pushing 10-20 years and are still non-managerial. The program would likely fall apart for a few months if all of them just left at once, so I wouldn't say they lack value. :)

A manager with a scant 3 years of experience undervaluing a "solo dev" (whatever that is, unless you're talking about 1-person companies) with 10 years experience sounds like a good setup for an episode of Silicon Valley.

There are a whole lot of senior developers who've done a turn as lead/manager, only to go back to dev. It's not for everyone, and promoting this narrative that it should be everyones goal is kind of shallow.

If you're at the same company, generally the answer is obvious. Overly bored with the tech and internal politics. Probably, brushing up on interviews to escape.

I don't think answer that question honestly takes you out of contention. You simply state what your career goals are in broad strokes. Say 'I want to advance to a position to mentor others' or 'I want to be managing a small team' or things like that. This helps make things clear as to what you are looking for in the future. If I know you want to be a team lead, manager, whatever that helps shape the job and your career growth, and maybe keep you around longer. Or if I know you have no interest in management, you don't get forced into it and leave earlier than you would have.

Obviously shouldn't answer and say 'I want to work at Major Competitor X, doing Y', but being honest is good for everyone involved. Everyone knows people stick around for maybe a few years and move on. 5 years is a long time, and you may not be (probably not) working for the same company, but its a question about character, motivation, and goals. Not company loyalty or some sort of trap question.

I would try to keep the audience and context in mind and answer accordingly.

It depends on whether you are interviewing the job or the job is interviewing you :)

Almost by definition, if you think being able to answer a question is important, and you only hire people who know how to answer that question, you will always think it's important.

>if an interviewee does not know how to answer a question, they don't know why that question is important or where that type of problem is applied

I would argue that not knowing answer and not knowing question are two different things. I may not know answer to a questions, but know that the question exists. I personally am comfortable not remembering things that I can easily look up.

I don't know the answer but I know how to get it.

I've thought of interviewing as finding the intersection of something to have a discussion about. Tech is so broad and deep it's possible that two people will have minimal overlap. It's nice to find some common ground you can discuss in depth.

If both the interviewer and interviewee have overlap they can discuss then the interview is most productive.

"15% dislike academic CS (and think that talking about CS is a sign that a candidate will not be productive)"

That seems really weird. Academic CS isn't really necessary for most programming jobs, but I can't see how it would ever be a detriment.

I had a guy work for me that was very academic minded regarding programming. He had a hard time letting go of the "pure" way of doing things and taking a good/practical approach to just getting stuff done. It can definitely be, but won't always be, a detriment in my experience.

That's a mindset flaw, not one that can be attributed to having CS knowledge, or enjoying talking about academic CS.

This is exactly true. There are certain things about which you absolutely need to be anal about being pure, such as scientific experiments and theoretical deductions/proofs. And there are others where you need to be more practical. This one person seems to have been applying the same mentality to both, which is a personal flaw, not solely a result of having more academic credentials/experience.

I like how you discriminate against CS knowledge in general because a single person couldn't get stuff done.

What you've described is someone who follow some practices religiously. I don't see any correlation between people like that and ones with academic background.

I'm afraid academia may produce religion. After all, it's a place where they show you a bunch of tricks, you practice them in simplified applications and then risk leaving with a belief that you actually understand shit.

A recent example: https://news.ycombinator.com/item?id=14267555

I've worked at places where well-meaning 'just get stuff done' people created mountains of technical debt that hobbled the project in the long term. Made them look great, though.

Great programmers should hopefully recognize and implement a proper balance between practicality and rigor, as necessary. They should also exhibit self-awareness and self-restraint, which sounds like the real problem for the guy you mention.

>I've worked at places where well-meaning 'just get stuff done' people created mountains of technical debt that hobbled the project in the long term. Made them look great, though.

Those people were just getting a bit better at building things. They were building experience. It just so happens you found their bad code that they'll realize was terrible a few years from now.

We can't simultaneously hate our own previous code and expect everyone else to delivery perfectly when they first start (or even at mid-level).

anecdotally, I see a higher-than-normal correlation between those who are "more theoretical" and those who cannot adapt to change or differing situations.

There's also an immense cost to not having CS fundamentals in terms of performance and wasted time; I don't want to pay someone to try and solve a version of the halting problem, or matching opening and closing delimiters with regexes.

>Academic CS isn't really necessary for most programming jobs

Knowing how to code quicksort or reverse a binary tree isn't, but an understanding of normalization definitely is...

No. Source: I have worked for a financially successful company whose database was designed without any notion of foreign keys or surrogate keys. They also had serialized Java objects in database blobs by accident.

I've worked for several like that. The blob antipattern isn't uncommon.

I always use normalization/database design as an interview question. I firmly believe that good questions about database design can really identify the candidate's skills and weaknesses in this area:

- it works well sitting around a whiteboard, which is a nice friendly dynamic that emulates how we really work

- it doesn't require detailed/arcane recollection of specifics, is more conceptual. Who wants to fail someone because they don't remember a parameter or some specific details of how TreeMaps work?

- it can't be faked - no-one can talk their way through a database design discussion unless they really have an aptitude for it

- its interesting. Who doesn't want to spitball around, say, modelling what Uber's database might look like behind the scenes.

I would say there are plenty of decent programmers around who can't (or haven't) done any real database design work, but if its a core skill (SaaS app development) that you really need, then its an easy one to detect at interview.

Too much focus on academic topics is likely a sign of inexperience. If the candidate had good real-world experience, they could (and should) speak of practicalities instead of theory. If the explanation involves a reasonable amount of theory, that's all well and good, but too much is an orange flag because it signals inexperience. It may also signal a propensity for dwelling on theory instead of Getting Things Done(TM). Companies almost always favor iterative execution of imperfect systems over protracted theorizing/analysis.

Depending on context and delivery, overfocus on academics could also be interpreted as snobbery, giving an impression that the candidate is difficult to work with, or that they think less of potential colleagues who don't have the same formal training.

Huh? Being experienced does not mean you stop liking the theory. If nothing else, more experience makes you appretiate a lot of it more.

It's not about "liking" theory. It's about talking about issues as theory as opposed to practicalities. You can like theory as much as you want, but if you don't talk about the real application, people will assume you've gone to class but haven't put it into practice.

It is interviewers job to get rid of illogical assumptions like this. This particular assumption is odd, experienced people often moves to talk about theory - sometime literally because they expect interviewers to be interested in that.

I don't think it's odd. The article says that 15% of Triplebyte's clients dislike it when things get too theoretical. Absent other signals, veering deep into theory on the assumption that interviewers generally like that is clearly the wrong idea.

My experience is that most people are happy to leave school behind, and they invoke theoretical concerns only when they are applicable and relevant. People want to hear about you've built and what you've done, they don't want their compsci textbook recited back to them.

If a candidate only has heavily academic projects, or can only discuss projects in heavily academic terms, interviewers may assume that the candidate is either a student or a researcher, and in either case, that person is probably not a fit for a conventional programming job (but they may be a great fit for a much cooler job in R&D somewhere).

Some companies really value an intensive academic background, but not all or even most of them (the article says that only 40% "need to see" academic CS skills, 15% actively dislike overt CS discussion, and that leaves 45% that didn't respond or don't care or whatever -- 60/40 split that talking less CS is at least harmless if not beneficial).

The takeaway, IMO, is to keep things at the high level, and stay focused on real, practical experience until prompted otherwise.

Guess I Am lucky that I meet good programmers who were not above theory and continually keep themselves informed. Contrary to what you write, there is no dichotomy between the two and your coding skills are not harmed by learning theory or keeping interest in it.

I also think that if knowing something harms me in the eyes of the company, then I do not want to work there. Such environment must be demotivating for learning. And i need to continuously learn. It is ok if they don't care and it is awesome if they are interested in different things that me (I can learn from them). Especially if we are talking about small companies where people who interview you reflect wider culture. Working in environment where people sort of punish you for knowing things they don't know is road to mediocrity. I want to work with collegues that will motivate me to learn - that is super important in length term.

So that leaves 85% of companies where I can talk about theory and not be harmed at all.

You're continuing to conflate knowing theory with focusing too much on theory in the interview process. I don't think anyone begrudges candidates for knowing theory. It's when their answers are primarily theoretical instead of primarily practical that the issue comes into play.

> You're continuing to conflate knowing theory with focusing too much on theory in the interview process.

That sounds like the social skills issue. If the interviewer signals change of topic is needed and interviewee does not follow, you definitely know the person has limitation in the social skills area. Also, if the person can not talk about practical aspect, they can't.

However, "15% actively dislike overt CS discussion" does not suggest companies that merely wanted to talk about practical aspect and could not. It suggest companies that are hostile towards people who attempt to talk about theory - talk about it too much and you are primary theoretical despite being also capable practicaly.

In any case, I do not want to work somewhere where I am expected not to talk about what I know (beyond the normal "don't have long monologues they areally not interested in"). I want my collegues to talk about what they know too.

I have a CS degree from CMU, and I've only run into that sort of attitude once. The interviewer nit-picked my solution to some character array manipulation question. I didn't get an offer. The interviewer wrote some snarky comment on his Twitter the day after my interview; something about "the difference between a computer scientist and engineer". Six months later his company was bought and chopped up. He got laid off. Karma's a bitch.

Conversely, I've been hired, without a formal interview, twice merely for having gone to CMU. I guess it can work out either way.

In senior positions CS knowledge is less relevant day-to-day, because your experience overrides a lot of that theoretical stuff. However, whenever I apply to companies, the questions about basic CS usually result in me turning down interviews. It would be a poor use of my time to memorize that stuff again only to not use it beyond the interview.

I guess it depends on whether you want to agree to that stupid system. When I interview now, I rarely ever prepare, and I'm completely honest with the extent of how much I remember and don't with the interviewers. If, on learning this, they think of it as a deal breaker, I think its a good thing for both of us to not be working together. However, if they don't judge me, but are willing to provide that information and work with me on the solution, I am fine with it.

I had a really bad experience with this recently, where I had a final interview which went really well in 3 out of 4 interviews. In one of them the interviewer was expecting a very specific solution, and I believe that I said something like, sorry I don't exactly remember how tries work, and I believe that is what resulted in no offer.

At first I was disappointed greatly, since it seemed like a great opportunity, and I had a great interview and experience with the other 3. But in hindsight, I realized I would really not enjoy working with this person every day.

It’s a loss for any interviewer to let pedigree bias them one way or another. If you’re willing to be interviewed, then they’ve got a pretty good opportunity to make a much more informed decision than letting their bias do it for them.

I'm not sure I'd want to work at a place that was hiring me based on just the school I went to. Similarly, I don't want to work at places that screen on irrelevant "CS fundamentals", because some of my favorite developers to work with would never be hired there.

I think it's largely just human nature. People don't know what they don't know, and those who "make it" without college are often particularly disdainful of academics. They may also be hesitant to hire those with a deeper background (which is a general phenomenon in many areas). In many fields, the percentage who can "make it" without higher education may be higher, but it's certainly do-able in our industry -- especially if they stick to a certain kind of software. Someone in this position may develop a real knack for avoiding hard problems (in the computational complexity sense) without ever quite realizing that is what they are doing. I almost didn't go to college, since I thought I knew it all, and had plenty of offers when I graduated high school. One of the most general things I learned was how much useful stuff there was to know, that I would have never fathomed.

In my experience, people are often hesitant to hire people who are higher on some (perhaps tacit) measure of nerdiness. Only when people want to win badly enough and it's clear that others are doing better, will many people start questioning such biases.

It's clearly a detriment to the hiring process: a bunch of academic CS minded types are the ones who set these absurd CS trivia interview questions.

I think we all know people who are very detail oriented and will argue ad infinitum about minutiae, but don't actually get a lot done in a speedy fashion.

I imagine they correlate an over interest in CS concepts with said group.

Since people in this thread are trotting out the old "we have to FizzBuzz people to weed out the millions of fake coders" argument:

Remember that FizzBuzz was originally proposed as a response to the number of CS degree holders who allegedly could not code their way out of a paper bag.

I don't care if someone can derive the hierarchy of Turing degrees from first principles. I care if they can do the job I'm trying to hire for.

We've had success with take-home coding challenges.

After 1-2 technical phone screens, we send applicants a coding challenge for which they have 48 hours to complete. The coding challenge is representative of some of the work we do (e.g. Given our API, create a D3 visualization of X) and should take applicants several hours to complete.

In submissions we look at everything from overall program structure and approach to applicants' attention to detail and usage of good industry practices.

This coding challenge quickly weeds out applicants that interview well but code poorly and is an efficient way for us to see the quality of work we can expect from them (and conversely, the type of work they can expect to do with us).

After a successful challenge, we bring applicants in to meet the team. At this point, it's largely just about culture fit.

Do you specifically hire D3 developers? This sounds like a pretty difficult "coding challenge" for a non-D3 guy, let alone someone who may not even primarily use javascript. Having used D3 sparingly before i've found it something that requires a fair amount of upfront domain knowledge to get much done beyond something extremely simple (unless of course you copy an existing example, which is what I always did). This seems unreasonably difficult for someone with no prior knowledge to accomplish in a 48 hour window whilst working etc -- you may find you are inadvertently optimizing your interview process for the unemployed with this kind of time limit / coding investment.

No, we're not specifically looking for experienced D3 devs. That particular challenge can be completed with the simplest D3 viz if desired (of which there are hundreds of examples online) and doesn't even require prior D3 experience (I didn't have any when I applied). We do look for a solid working knowledge of JS, however, and an ability to learn new tools quickly.

The 48 hour time limit is a relatively arbitrary timebox and isn't a hard limit; we value quality over speed.

I've performed interviews and phone screens for years and out of all the techniques I've tried all of them typically end up with me thinking I'm doing it wrong. Basic domain questions, problem solving, white boarding, white boarding along side the person helping them out, coding in an IDE, take home projects. As of today, in Kansas City, I'm looking at around 4% pass rate, and close to 90% utter face palming failures.

I check my ego at the door, I try to make the person as comfortable as possible, small talk, help them out as much as I can, provide them with answers or explanations when they get something wrong. I've never asked an algorithm or data structure question, I've rarely asked a patterns question. Most people can't answer basic every day questions.

Overtime my goal in interviews turned into making sure the interviewee left with more domain knowledge than when they came in hoping it will help them do better somewhere else.

Ended up creating a pre-screening service that asks the same basic questions and in the end provides the user with a study guide based on how they answered. Currently piloting with two local recruiting companies and I'm finding the same similar statistics of failure/success. Maybe that makes me the issue :)

My numbers were about the same when I was interviewing for JavaScript developers years ago at Travelocity. The only 1 skill I absolutely could not bend on was the ability to walk the DOM. Only 4 out of the 22 people I interviewed could do this and only two of them could do it well.

Strangely enough I warned candidates days before the interview of what I was looking for and still almost nobody bothered to brush up on the couple of basic methods needed to do this 1 task.

You know the quote: if you encounter an asshole first thing in the morning, you encountered an asshole. If you encounter assholes constantly all day long, it's probable you're the one who's the asshole.

If you have a 90% rate of "utter face palming failures", the problem isn't with the people you're interviewing, it's with the interviewer and/or the interviewing process.

That's what I keep questioning and why I've tried various methods. After screening resumes along with coworkers, at various companies I've worked at, we'll pre-screen or not, bring them in and ask them questions from their resume.

Resume says 'I'm an expert in SQL'. Great, lets start some every day foundational questions. What is the difference between an inner join and an outer join. Why might we use a varchar instead of a char data type? Why do we use indexes? What is the purpose of a foreign key? A good majority of the candidates can only answer the join question.

'10 years experience, senior dev in <main language>' can you write a function that returns the largest number in an unsorted array? Most struggle to even pseudo code it. Tell me something, anything about interfaces. Basic security question on SQL injection/xss/csrf/hashing/encryption. Without a doubt most have only heard of SQL injection and then they get that wrong. Most say hashing and encryption are the same thing.

I do blame myself, the failure rates are incredibly high. But when resumes looks great and onsite can't pseudo code a simple loop, or answer basic questions about something they claim expertise in, what can you do? Require and call references to make sure the the lead dev with 12 years experience and decent companies isn't lying? Because the result of most of my interviews appears that people inflate their resumes and flat out lie.

So let me ask you an "easy" question: for your interview questions, are you consistent in what you ask, and do you have a consistent approach to evaluating a candidate's response?

I ask this because some of your "foundational" questions are not as simple as you might think (I could talk for literally hours about XSS and the only conclusion I could give you would be "there's no sure-fire way to prevent it", for example), and others border on pop-quiz material, which is not really a great way to evaluate someone.

Another question that's important to ask: you say you're working with recruiting companies. Have you tried not doing that? Recruiting companies don't exist to find you qualified people, they exist to spam you with résumés. Recruiting companies have been known to flat-out alter résumés to make them better match the set of keywords you said you wanted. And my own experience is that recruiting companies are the source of a huge percentage of headaches in hiring. Cutting out the middlemen can and likely will drastically increase your success rate.

I try to be consistent with my questions, but it depends on how well the candidate is answering them as to the depth I go into a subject. Generally I ask the candidates about previous projects and work my questions into the conversations. It usually isn't a rapid fire pop-quiz type interview.

Tell me about a project you had the most fun working on. You tell me you started working on a SPA in Angular 2 and I ask if you've heard of some common attacks like XSS or CSRF. You say yes to both and I ask if you had any preventive measures in place or how you would go about trying to prevent them (I'd stop you if you kept going on and on). We'll work our way through the projects stack best we can in the same manner.

We generally used recruiters at all the companies I worked at due to lack of applications. I agree they're part of the issue. I also phone screen for recruiters so at least some of them are doing their due diligence (no idea how they act on my reviews though).

As someone with an actual accredited engineering degree, I find it bizarre that Silicon Valley has decided that the term "engineer" doesn't even need to be qualified anymore. Even without getting into the flame war over whether "software engineers" are real engineers or not, there's a very large pool of engineers to which this post doesn't apply at all. I can't be the only one in the latter pool that reads HN, can I?

I used to be reluctant to call myself an engineer. I don't actually get to do too much engineering in my work. But I've seen too many less qualified people calling themselves engineers so I might as well... Besides, my degree is from the Engineering faculty at my university rather than Science where the CS/SE degrees are. At least that's how I justify it to myself.

You're not the only one. I have a Master's and it was hard. I went for my engineering degrees after more than a decade as a technician and programmer in telecom. It's a world of difference. A cert from Microsoft or Cisco or whomever doesn't compare. We earn the title, they enjoy it.

Where I work, we'd be totally happy to ask interviewees how to reverse a binary search tree. As soon as an actual developer creates a pull request that does as such. Which hasn't happened yet. So we don't ask about things like that. And everyone seems happier.

I don't get it. Programmers are surrounded by trees. The filesystem, your code's AST, dependency trees, the HTML DOM, any JSON ... why do people consider it so unreasonable to ask a question about simple manipulation of a well-understood tree data structure?

It would be reasonable to ask tree related questions. If the questions falls within a normal use case. Traversing a filesystem is something you should be familiar with. Reversing a binary tree. No

Unless you're writing a database or browser engine, you will never have to implement a binary tree. So it is effectively trivia knowledge. It's like asking how the x86 CPU architecture works, because it's the platform on which most code runs.

Because it implicitly selects for inexperienced/mediocre candidates over experienced ones.

Even among people who did a degree that involved implementing these algorithms, 99% or more will never do it again once they set foot off their campus, because they'll just use a library that already has it implemented. Then they'll free up room in their brain's working set for the stuff they actually do have to implement. And the longer they go like that, the longer it'll take and the more difficult it'll be to find where all those tree algorithms got paged out to in their brain, and the worse they'll look on an interview which consists solely of asking for that stuff.

Meanwhile, the inexperienced and likely unqualified (given what people claim about the average CS graduate) person who just walked out of their college graduation still has it fresh and ready to regurgitate onto your whiteboard, and passes with flying colors: great technical skills, great "fundamentals", A+++++ hire ASAP!

The solution to this is to stop using proxies for the thing you want to know, and start actually testing for the thing you want to know.

Here's a concept I had a question about that I don't think gets discussed that often...

If a company were to offer a base salary at a mere third of the Google base salary for the same job, why should an applicant have to copy Google's depth of computer science abstraction in the coding interview?

Is the onus on the developer for even entertaining such a demanding question at a low pay, or does the current economy favor employers in such a way that developers have no other choice? Or rather, is the knowledge of available jobs asymmetric enough to where employers can pay pennies, piss, and peanuts for A-level employees?

I don't see a connection between pay and the interview process. Should the company also make their chairs one-third the height? Have one third the whiteboard marker colors? Feed you one-third of a lunch?

My willingness to jump through hoops is proportionate to how much I want to work with you. If you pay me well and treat me well, I want to work with you. If you're trying to hire me for 50% of market, my tolerance is much lower.

Then the filter would be "The skills of a Google Engineer, but can't get in there so might work here". All others need not apply?

well, if you interview for a nurse position, you shouldn't have to pass the doctor interview.

Oh. I thought the statement was "for the same job".

Honestly drop the question/answer aspect for technical checkouts. Offer a problem then work with them to solve it. The most important aspects are two fold, What questions do they ask, and now that you are being cooperative and a team mate, how do they interact with you.

EDIT: But during the entire process keep in mind they are hopefully interviewing your company. Far too many companies forget this aspect and the entire process has become lopsided in favor of companies making candidates jump through hoops.

Looking for a job has become a full time job. In depth technical discussions about the technicalities, then about technical quirks, onsite visits and time spend on researching what on earth the company is doing and what their project is about, then the curse of the industry - take home assignments. Once I signed my last employment contract I said to myself "I'm exhausted, there is no way I'll move a finger during the first months of employment in this company".

Also I learned to be more picky. "Want to see my code and apps? Show me your code, show me how you work!" Eclipse required? Bad workflows with weird limitations? "This is how we do it here, it works for us" attitude? High discipline environment with daily standups? Cheapskate workplace/outsourcing center? Sorry, I'm passing.

>"After an engineer passes our process, they go straight to the final interview at companies we work with (including Apple, Facebook, Dropbox and Stripe)."

Can anyone from any of these companies confirm this? This strikes me as really odd. So only one employee from FB, Apple, Dropbox, and Stripe needs to interview a candidate that Tripplebyte say is a "Go"? I'm having a hard time believing that.

I don't think "the final interview" means a single person. I took it to mean the onsite round, with however many people that entails, skipping the recruiter screen, phone screen, etc.

This interpretation is correct; they skip to onsite, but get a full onsite. (Source: we interview triplebyte candidates regularly)

how is the quality of the candidates coming thru the triplebyte pipeline compared to recruiters, or direct site applications?

I'm considering using triplebyte, but also want to control where my applications are sent to.

They are of similar quality to candidates that get through our normal screening process (direct applications). It is best to be very specific about what types of candidates you are looking for so they can match as well as possible. We're happy with them in general.

So why does it matter whether the first round of interviews is done by this company or the actual company doing the hiring?

I would argue that as a candidate I would rather interview with as many of my potential coworkers as possible. Interviewing is a two way street. The more exposure to the actual people I would be working with allows me to make a better-informed decision. How is reducing my exposure better for me the candidate? It seems its really better for Tripplebyte though and maybe the company looking to hire if they are short-staffed but not the candidate.

> So why does it matter whether the first round of interviews is done by this company or the actual company doing the hiring?

I think the idea is that if you do the first round with Triplebyte instead of with Apple, Facebook, Dropbox, or Stripe you are effectively doing the first round with Apple, Facebook, Dropbox, AND Stripe.

>"... you are effectively doing the first round with Apple, Facebook, Dropbox, AND Stripe."

Except that I am not. By doing the first rounds with Tripplebyte I have squandered an opportunity to learn about and form an opinion about the people and team I might be working with in the future. The more answers I can get to my questions the better informed I am.

I guess I don't understand this whole "we will fast track you" as a value proposition. The only efficiency gain("fast tracking") is gotten by the company looking to hire. I myself am still subject to the same length of the interview sequence. Only one is with Tripplebyte and one is with the company doing the hiring instead of two or more with the company doing the hiring. And in the process I have given up the opportunity to have more exposure to meet and talk to the actual team I would be working with.

Should I be so grateful for an opportunity to work for "Apple, Facebook, Dropbox, AND Stripe"(Tripplebyte seems to beat this drum loudly) that it doesn't matter whether I think it would be a fit for me on a personal, cultural and professional level?

Assuming you don’t pass 100% of on site interviews or you don’t take the first offer immediately, triplebyte will save you some amount of time. Beyond the force multiplier for pre-onsite interviews, they also can predict which companies you will have a higher success rate with for your on site. It’s not a guaranteed slam dunk for every single person (and you still have to pass their process) but I think the value proposition is pretty obvious to most people.

>"Assuming you don’t pass 100% of on site interviews or you don’t take the first offer immediately, triplebyte will save you some amount of time."

How exactly do they save me time? Can you elaborate?

>"Beyond the force multiplier for pre-onsite interviews, they also can predict which companies you will have a higher success rate with for your on site."

What is the "force multiplier" here?

Again this is a job where you will spend many hours of your life, should the only concern be the one whose interview I will have the highest success rate with?

>"... but I think the value proposition is pretty obvious to most people."

Again what is that? It's not obvious to me. I am asking sincerely.

Let me try to answer: Say you plan to interview at Apple, Facebook, Dropbox, and Stripe. Lets say each company requires 2 rounds of interviews. Without these guys, you're doing two rounds at each company = 8 interviews. With these guys, you do one round with them, and one round with each of the companies = 5 interviews. If that's how Triplebyte works, it would be a huge time saver.

Does every engineer unquestionably want to work for any and all 4 of those companies? Aren't these very different companies? Is there any significant commonality here other than Tripplebyte has some connection at all of these?

Pick N other companies, then. As long as TripleByte has this kind of relationship with at least 2 of them, it's a time saver. I could easily see myself interviewing for 20 companies in a single job search. Needing only one "pre-screen" for all of them would be nothing short of miraculous.

Let's say you avoided interviewing with 4 future team mates from company Company A by using Tripplebyte.

And come to find out most of those 4 people you didn't have a chance to interact with during the interview process are completely toxic and unprofessional. You only find this out after you have accepted the job.

If you find yourself looking for a new job again in month then Tripplebyte hasn't really saved you any time have they? I am not sure where this idea comes form that just because the company is "Apple, Facebook, Dropbox and Stripe" does not automatically mean that everyone on every team is a great personally or one you would want to work with.

When I interview with a company and I don't think I am alone in this I am also interviewing the company. I am thinking "are these people I could work with?" The novelty of working on Big S.V. Company eventually wears off and if it's a bad fit for you on personality or culture then you are either going to be unhappy, looking for a new job or both. I and I imagine many others are OK with a slower process if it means being able to make a better informed decision.

I'm with you on this. Of all of the teams at Apple/Facebook/Stripe etc I can guarantee there are at least 25% I would not want to be on, 30% I might not want to be on, 30% I might want to be on and 15% I would definitely want to be on.

So now I need to start from scratch to get all of the information I'd need to make the decision of what position I am interviewing for fits. What makes this doubly difficult is I've been "fast tracked" to the interview process where those discussions aren't expected.

The more I read about triplebyte the more I think their heart is in the right place but they are creating a market for lemons.

If triplebyte can create a system where the final interview is a cultural fit interview of no more than 2 hours and both sides feel they have enough information to proceed, then they are onto something.

Interesting, someone clarified below that it is indeed final round. I would take it to mean 'the final interview', singular, as written. Seems misleading.

I think most people would agree that "final interview" is singular and a "round of interviews" is plural. It is indeed misleading.

Sometimes I ask candidates to come up with a question for the interview, something that would allow them to put their best foot forward.

Any thoughts on that?

I've never seen it done, but it sounds like there's potential.

How has it played out when you've done that?

Not well. Candidates get disoriented and can't come up with anything relevant. It's a very small dataset though.

Never heard of it but what a great idea!

The question is really under-constrained and it's not clear what you are trying to figure out.

I ask things like, "what's the most interesting bug you've encountered?" or "what have you been excited about learning recently?" to try to find a topic we can have a conversation about in a domain they are familiar with and interested in.

Oh, I forgot to add - I actually ask for a coding task they think will put them in the best light. Like, "hey, I can balance a binary tree, let me show you". In my mind if someone learned something like this very well they show their learning potential, and complexity of the task will tell me their self-assessed level of development.

>I'm arguing for asking easy questions, but then judging fairly harshly.

I agree with this. Better yet ask a multi-part question that has an easy first part and a harder follow on.

Interesting read from the POV of the staffing agency and hiring company. But why should I jump through those hoops when I don't even know if I'm interested in the job? I really think the hiring companies need to sell me on them before they ask me to spend an hour of my time.

I agree. I had an interview yesterday where they passed, and the reasoning was "You didn't seem interested in the job." And I sort of think that's hilarious, because nobody tried to sell me on the job. Nobody told me exactly what the job would entail. I never met the hiring manager, even though I was on-site for 5 hours. And clearly they weren't interested in me... It just seems weird that I have to be jizzing my pants to work for them, but they don't even have to try to sell me on the role.

And this goes doubly true for any company that reaches out to me first. Like, if I apply to your company, then yeah, you have an expectation of me being interested in the company. But if you reach out to me, then maybe you're the one who needs to sell yourself to me.

I don't really get that attitude by employers. Whenever I'm interviewing candidates, I really try to sell them on the role. I try to tell them what's awesome about it, why they should want to work here over literally anywhere else in the world, and why you WANT to sit next to us for the next six months and work with me and my team on projects. Part of my job as an interviewer is to not only gauge your ability and interest, but to convince you that we're the best place for you to be.

Or... do interviewers really think I selected their company is the only company I even considered talking to? We're engineers. We have options. You (the company) have to meet me at least halfway.

Incidentally, I've often wanted to ask my interviewers to solve a tech challenge. I don't want to join a company where the team is incompetent and I'm constantly cleaning up after them. Oddly, that's never a consideration that companies allow candidates... ;)

One thing I find interesting is the absolute conviction that every interviewer has that they got it right when the reject a candidate.

No "reject" decision can ever be reconsidered or questioned by anyone, ever.

If I were in charge of a giant company I'd hire every 50,000th applicant or whatever, just so we had a control to see if our interview process was worth the time.

At my company our engineers defined the following process.

First, we defined the skills we're looking for i.e. Programming / SysAdmin / Cybersecurity.

Our process goes as follows:

1. We ask candidates to answer a quizz by phone, with questions in the 3 chosen fields. Duration = 1h.

2. We ask candidates to solve remotely with Google Docs 5 real-world problems asking for skills in Algorithmics, Data Modeling, Object-Oriented Programming, Software Design, and Parsing. We also add a bonus question like "how many people can get into a train". Duration = 1h.

3. We have a HR meeting at our office with a non-technical member of the team. Duration = 1h.

The quizz helps us to remove of our list the candidates that do not have the technical culture we're looking for, and to see what they already have worked on.

The 5 problems helps us to see how the candidates can code, the bonus question helps us to see how they approach new issues and manage their stress.

The final HR meeting is very typical: we try to see if we would be happy to take a 6 hours long flight with the candidate.

Most candidates do not go through first step, and roughly half on them do not go through second step.

This simple process definitely helped us to reduce the time spent on hiring and to make the really good candidates shine.

> how many people can get into a train

Can't speak for everyone, but I'd turn away and walked out the instant I hear this or similar BS (which is totally unrelated to "approaching new problems" or "stress management" in programming at least)

You're too forgiving. I turn down interviews when they list "linked lists, hashing, breadth/depth first search" on their study guide. I've never had to write a linked list EVER in my career, don't fucking bother me with that shit. Also, if the position is in a language where linked lists would be stupid (i.e. python), then I definitely reject that company.

I can explain what everyone of them is, and why you'd want to use them. But I've never written any one of them and I'm not going to spend the time to memorize something I can look up in my CLR book or stackoverflow.

It's embarrassing that tech interviewing is still stuck in a 1990s mindset.

I've written at least 5 linked lists in the past year. They're quite useful.

What were you using them for?

Not sure about GP, but very recently, an open-addressing hashtable that could be traversed in both reverse modification and reverse insertion time order. There are actually some interesting subtleties when doing this for open-addressing hashtables (i.e., entries (and pointers to them) move around when the table rehashes).

Five different implementations of linked lists?

Yes. To be fair, this is the first time in my life I can clearly remember implementing a number of linked lists.

I'm pretty sure my lifetime total is not much greater than 5.

(Singly linked) lists have many advantages (e.g. useful operations on them are side-effect free), but whether they should be used depends upon which problems you solve and which languages you use. It's unsurprising they're used all the time in Lisp, and functional programming languages generally. I'd advise against their use in C or other languages without garbage collectors. They're rarely needed in languages (such as Python and Java) which have vectors and hash tables as basic data structures. I've never found a need for doubly linked lists in any language, for any problem.

So you'll either have them built in to the language you'll be using, the company's using the wrong language, or you'll be tackling problems where they're not really needed.

What language would you recommend using a singly-linked list in, aside from Lisp? Why is it any less valid in non-GC'd languages? (Aside from memory management in C is fraught with error, but that's a problem not unique to singly-linked lists.)

I've never looked (at least, that hard) at the language; it was always a question of "is this the appropriate data structure for this task?". Linked-lists have somewhat peculiar time complexities, but occasionally those line up with what you need. (Usually a vector is also as good, time-complexity-wise, and wins out by being contiguous. But if you need O(1) removal…)

Linked lists make way more sense in languages with bump-pointer allocating garbage collectors specifically. In those languages (e.g Haskell, OCaml, Java) allocation is really cheap and they preserve locality so allocating all the links isn't as much of a penalty.

Also if you don't have a garbage collector you have to do reference counting, and if you want parallelism your reference counting has to be atomic, which when you're sharing a lot of linked list nodes, is another large penalty.

> Linked lists make way more sense in languages with bump-pointer allocating garbage collectors specifically. In those languages (e.g Haskell, OCaml, Java) allocation is really cheap and they preserve locality so allocating all the links isn't as much of a penalty.

Such allocation is possible in languages such as C++/Rust/C as well. (Though to how much the standard library in each supports you may vary.)

My point was that at least I choose linked lists for the O(1) node insertion/removals in addition to maintaining a sequence of items, a trait pretty much unique to linked lists. It's a matter of whether or not the order notation beats out any additional allocation time and cache effects (Vec/std::vector/continuous-memory lists being more friendly towards cache)

> Also if you don't have a garbage collector you have to do reference counting, and if you want parallelism your reference counting has to be atomic, which when you're sharing a lot of linked list nodes, is another large penalty.

Neither Rust nor C++'s linked list implementations require ref counting, and neither have a GC. A node is essentially,

  struct LLNode<T> {
    LLNode *next;
    LLNode *prev;  // if doubly linked
    T item;
  }

(the above is pseudo-syntax, adjust appropriately) Rust, for example, knows the linked list is only ever owned by a single person. (Barring either the list or the items being wrapped in Rc/Arc/Mutex/etc., but in that case, of course it is, but that's yet different and doesn't really impinge upon a linked list's general usefulness)

Yah I know normal linked lists are totally possible in all languages. I should have specified I was talking about the use case of sharing data through immutable singly-linked lists.

In these languages you use them specifically because you can prepend without worrying about other things still holding a reference, even in other threads. In Rust you'd need Arc to do that.

Also yes you can use custom allocators in Rust/C++, but then you have to have everything have the same lifetime. With GC'd functional languages you don't because the GC will move them to the old generation if necessary.

I'm mainly talking about the empirical question of why Lisp/Haskell/OCaml use linked lists so much, but in other languages they're rare (e.g I see/use them less than heaps). In most cases where you want fast prepend you just append to a vector and just treat it as reversed or reverse it after.

For me it is opposite. If the company gives reasonably sized study guide, I am cool with it. I also implemented linked list like structures (it was not linked list but similar). You don't need to memorize it to suceed, the construction is pretty logical.

We ask candidates to solve remotely with Google Docs

Google Docs are horrible coding environments. In fact, anything besides the candidate's native editing environment makes for an obtuse and off-putting experience, all around. I mean, yeah, the candidate can suck it up and make it work if they felt they "had" to. But on balance it's just unnecessary mental gymnastics -- and completely avoidable source of awkwardness and all around unpleasantness in the interview experience.

> In fact, anything besides the candidate's native editing environment makes for an obtuse and off-putting experience, all around.

And how would I use that, as an interviewer, in a phone screen, realistically?

(I don't believe that I could — reliably — get them into a Google Hangout, for example.)

(We used https://coderpad.io/ ; it was alright, and certainly better than Google Docs.)

>"We ask candidates to answer a quizz by phone, with questions in the 3 chosen fields. Duration = 1h."

You ask trivia questions for a whole hour? That seems pretty gratuitous. I think most people would tire of that after about 15 minutes.

>"We also add a bonus question like "how many people can get into a train". Duration = 1h."

You ask Fermi questions? Between an hour of trivia questions and one "bonus" Fermi question your process sounds pretty horrible to me.

Interviewing candidates is a two way street. There are many candidates who would not want to work for a company that thought asking a candidate an hours worth of trivia questions and a bonus Fermi question was acceptable. For many this would be a red flag.

Are you serious?

How did the engineers who designed that process get hired? What was wrong with the process that landed you your existing team?

Lastly...why are the engineers coming up with the hiring process anyway? Is this common in other professions?

An alternative to an "interview" would be something more like an exam: have candidates try to solve a (relatively simple) problem in limited time, with or without external resources.

Candidates would be watched, would know they're being watched, but would not see themselves being watched, which would probably take a lot of stress out.

You could even test many candidates at the same time this way. And you could setup a system where each candidate could ask questions if needed, without being heard by other candidates (like in language learning classrooms).

This must have been tried? but I wonder why it's not even discussed in the post, as an alternative?

There are a couple of web services that offer this - a quick google search around should return some company names, such as www.interviewzen.com

Personally, I feel just as much stress knowing that the interviewer can review all my typos and mistakes in minute detail, leaving me to write code in my local editor and simply copy-paste working sections into the provided editor.

Amazon made use of a service similar to what you've described at one point, however they required access to your machine to verify you weren't "cheating" in the interview. You can read one of the (many) threads here:

https://news.ycombinator.com/item?id=13076073

My current employer does this.

You're sat in the conference room, given a machine that has a screen monitor program running, and a set of tasks to perform on the code. You are told where to find the code in the introduction on the sheet of instructions. You are allowed to use any resource via the computer (so nothing you've brought in like a phone or laptop is allowed; basically, all work has to be "shown"). Then...go.

Outside the room, the dev team watches the process. We want to see how you go about each problem, what resources you use, etc.

It's always surprising the number of people who go for an hour or more and do nothing. Just sit there (or seemingly sit there - there's no keyboard or mouse activity).

When I interviewed, they actually gave me an online test; the job was supposed to be for javascript, but the online was in PHP (which my prior position used, and I had been using for years). Passed that, and came in for the on-site part. I thought it was odd that they first tested me for PHP. Did a bit of face-to-face interview, then they wanted me to do the watched javascript portion (single page app).

I first sat and read the problems, formulated some possibilities in my head, then jumped in. 30 minutes later I was done, and popped open a new file to write something to that effect (figuring if they were watching, they'd come back in). I sat for several more minutes, then they came back.

"How's it going?"

"Alright...umm, hey, I'm done."

"No way! Are you serious? Are you sure?"

Odd...

"Yes."

Apparently they had never had anyone finish the test that quickly, and get it completely right - especially someone who also had PHP skills, and had already demo'd them in the first test, which was also done correctly. I found this rather odd, as neither test was that particularly difficult.

Apparently, though, it's not uncommon in our industry. I'm not sure what to make of that...

> 9. Make decisions based on max skill, not average or min skill

This is a strict specialist search - here, too you should make a decision depending on your requirements and future outlook: Do you need a generalist that can branch out and specialise, or strictly a specialist that only does the thing you need right now?

(Also let's not argue which is better, specialist or generalist - deep down, each of us already knows and made the choice ;) )

You can ask someone to solve a rubik cube in under a minute.

Or you can ask someone to beat a Go bot that is ranked over 5 kyu.

Or you can ask someone to find adjacent zeroes in a matrix of numbers in under 15 minutes.

All those exercises, however tricky, do not correlate much with how smart you are or what your programming abilities are.

Some engineers I interviewed got "offended" by ElonMusk-like questions and refused to answer :)

What type of questions do you mean?

Imagine a cottage with all 4 walls facing south. And there is a bear visiting it. What is the color of bear's fur?

The second you uttered that, I'd thank you for your time and walk out.

What on earth does that have to do with building (actually, in most cases, maintaining) a CRUD web app?

What does that have to do with n-tier web application architecture?

Is that a real question? I too would refuse that in a programmer interview, unless it was joke. Of course the answer is white - polar bear and you are at the north pole. Even as a puzzle question it's pretty lame. Still puzzling over the boiling bagels on Mars one...

If you really ask questions like that then you shouldn't be allowed to be involved in recruiting.

Why?

IMO, hiring is like a funnel. At the top of the funnel is the entire universe of people. At the bottom of the funnel is your hire. The goal is to filter out as many people as possible as quickly as possible. One key point is whether or not they can write code.

Keep in mind writing code is the BARE MINIMUM required for a job. If they can't do that, they shouldn't be on-site in the first place. If you have your candidate taking a day off work and taking a day worth of engineering talent off engineering-related tasks, and you're spending that time asking them to write code, you've failed to apply the filter higher up in the funnel.

Ideally, you verify their coding ability BEFORE you bring them into the office, whether via a tech phone call, live-coding session, or take-home assignment. Before the haters start saying "but they can fake it!", hold your horses. Once you've established that the candidate can code, when you bring them on-site, you quickly VERIFY your previous conclusions. You quickly VERIFY that the coding assignment you gave them was completed by them. This should take no more than 15-20 minutes. Maybe you just have them walk through a chunk of code. Maybe you ask them to recreate their answer. Something. Anything.

But if you send in three different engineers, and they all spend the entire hour asking the candidate to solve programming tasks, you're literally evaluating the BARE MINIMUM of the hiring requirements. And there's so, so, so much more to being a good engineer than writing code.

Outside of a QUICK verification, your on-site questions, imo, should NOT be writing code (or even pseudo-code). They should be about previous projects, architecture, personality, professionalism, punctuality, communication, teamwork, autonomy, and collaboration skills -- all the non-tangibles that answer the basic question of "Do I want to sit next to this person for the next 6-12 months?"

If your idea of a good interview is to ask someone to generate a calendar between two dates or sort a list or find the minimum integer in a list, you're completely, totally missing the point. You're evaluating the bare minimum to get them in the door -- and if you're doing that in-person, you're wasting your time. You should have filtered that out before they walked in the door.

Remember, these candidates are often lying to their bosses and calling in sick or using up valuable PTO to take an entire day off work to come sit in your office. Don't waste their time with redundant tests and things that you could have figured out before they walked in the door.

(And dear god, some of the questions in these comments are... awful... at evaluating anything except, I dunno, getting the right answer. Yikes.)

"There is not a single set of skills that define a good programer. ... No engineer can be strong in all off these areas."

The ability to spell is one of them.

I wouldn't use TripleByte to hire or get hired, but that was a well written article.

I agree with much of this. Some additional comments:

> Ask questions as close as possible to real work: This is achieved by making interview question as similar as possible to the job you want the candidate to do (or to the skill you're trying to measure).

This "or" thing in parentheses is really important. Interviews should not be "as similar as possible to the job." They should be all about the skill you want to measure. And there's quite a difference between these.

People can be trained, assuming you have time and resources.

The point of standard algorithm interviews is to measure a particular skill: intelligence. The belief behind this is smart people will tend to solve problems well in the real world. It's okay if they don't know all the skills they need; they can learn them. (And if there's a particularly challenging skill, then maybe you need to add that into the process.)

> Avoid hard questions: If a candidate solves a really hard question well, that tells you a lot about their skill. However, because the question is hard, most candidates will fail to solve it well. The expected amount of information gained from a question, then, is heavily impacted by the difficulty of the question. We find that the optimal difficulty level is significantly easier than most interviewers guess. This effect is amplified by the fact that there are two sources of signal when interviewing a candidate: whether they give the “correct” answer to a question, and their process / how easily they arrive at that answer.

You don't give examples of what qualifies as hard vs. easy, so it's a little difficult to judge this. But generally, I'd advocate for people asking harder questions.

When a question is easy, a little thing -- small point of confusion, etc -- can make a big difference in whether the candidate seems to have gotten to that answer well or not.

When a question is hard, this is when you can actually see a great differential between great vs. good vs. okay. You can offer help and see how effectively they pick up on that advice.

If your interviewers are just throwing out a question and then sitting back and seeing what happens, then yeah, candidates won't be able to tackle a hard question. But that's not what they should be doing.

To be clear here: hard does not mean "ask about some obscure data structure [skip lists, etc]." It means a challenging question involving common knowledge (hash tables, strings, arrays, maybe binary search trees).

> 5. Ask every candidate the same questions: [...] there is no justification for asking different questions to different candidates. If you evaluate different candidates for the same job in different ways, you are introducing noise.

I think this is overplayed a little.

If you're doing algorithm-style interviews, any question should be basically evaluating the same skill (problem solving skills). If I ask my favorite question and you ask your favorite question, but they're evaluating the same skill, then it doesn't really matter. There's nothing to make one question better than another. Why not let each interviewer ask the question that they like?

Now, if every interviewer asks a different question, you won't be able to give a well-defined metric on what good performance looks like for that problem. But that's okay. You can't really do that anyway.

When companies give well-defined metrics, they get too rigid. The candidate takes slightly longer to complete the code because the candidate thought things through more thoroughly, and now they're pushed into a lower performing bucket. And that's wrong.

These standardized metrics sound good in theory, but they don't work well in reality (for algorithm interviews, anyway).

So, we've gathered data on exactly these points over the last 2 years. And what we've found that the decision that gets the best signal is often not what feels most accurate to the interviewer.

For example, my guess before running these experiments would have been that simply looking at progress (how far a candidate gets through a problem) would be a bad measure of interview performance. I'd expect that things like style and how communication how careful a candidate was being would render pure progress a mad metric. However, when we compare pure progress to a subjective score the interviewer gives the candidate (ignoring specific reasons, does the interviewer think the candidate is a good engineer after the interview), we found that pure progress is more predictive of success at companies! (To be clear, we don't only look at progress at Tryplebyte. We've also found other things to be predictive.)

The same is true (among our candidates at least) for question difficulty. Easy, straightforward question (write a command line interface to store and retrieve key-value pairs) are more predictive of success at companies than question that try to target intelligence (calculate how much water would collect in a histogram)

Have you tried IQ testing instead of algorithm questions to measure intelligence?

There is no such thing as a single "intelligence". "IQ" is measuring at least three different forms of cognitive ability, is an inherently flawed and culturally biased tool and may be illegal to administer unless you can prove it's relevant to the work.

Additionally, what matters is not natural talent but the set of skills and techniques an individual has built up using those talents, compensating for their weaknesses and taking advantage of their strengths. I've met plenty of very, very smart people who flounder the first time they encounter a code base they can't hold entirely in their head at once: someone who was less "smart", with a smaller working memory, but who has been developing skill with abstraction and system metaphors since CS 101 is often a better actual developer.

The myth that developers need to be "smart" is pernicious, and the cause of most of the really horrific code bases in this industry.

> [IQ tests] may be illegal to administer unless you can prove it's relevant to the work.

Not any more than any other means of assessment on which outcomes differ on a protected axis of discrimination like race; yes, the rule was first articulated in a case involving a fairly blatant use of IQ tests to effect racial discrimination, but it is by no means restricted to IQ tests.

quiz != interview

Reply on news.ycombinator.com