Hacker News Reader

Comment by sethammons | Hacker News Reader

The rule of thumb I subscribed to: have the interviewer(s) do the interview. Give the candidate 3x time to do it that the in-house people took. Or, framed in reverse, if you want the assignment to take and n minutes or hours, give yourself n/3 to complete it, and however far you get is the bar for the assignment. Interviewees are stressed, maybe outside their domain, and other things. Make something that they can show you their value. The harder and correct thing to do is to find how they can help your org. It is easy to find what they don't know or to play stump the idiot.

View on HN

My high school physics teacher did the same. He would take the tests he prepared himself to check whether the scope was appropriate for the allocated time. The factor of 3 was spot on for this kind of estimation.

Rule of Threes.

I normally find 3 to be an insufficient factor for exams.

I normally choose divide by 10 for exams as I could do divide by 3 for exams from a different professor/class.

I know the material cold, know the test cold, and don't have to worry about hidden traps--so I expect to take less than 1/10th the time of a student.

We did this at my company. A developer at the level that that developer was applying for would do a task that took them 2-4 hours. We then gave the interviewee four days to complete the task with a recommended time of 2 days. We also always made sure to include the weekend in the time they had to complete the task as we know during the week time is precious.

A two day take-home coding test is absurd regardless of whether it's the weekend

I read the grandparent as: The task should take 2-4h and be done within 2-4 days is how I read it.

I’m still not sure why people expect it’s easy to fit this in and it’s the last thing you want to do after a hard day of work.

I’ve just started taking a full day off for programming exercises I do when interviewing. It allows me to give well thought through and tested solutions even if I make some small mistakes or misunderstandings. Crazy I know but necessary usually these days.

Except that they said it should take 2 days.

I think it’s badly worded and you can read how I parsed it in my comment, it’s a guess, but otherwise the rest of the comment makes zero sense.

They tell the interviewee that, the the given task actually takes 2-4 hours in-house.

This is why I only do these for exceptional offers. Like a studio I wanted to work at and a problem that seemed challenging. Otherwise it's a strict decline.

These days they don't give you pass even if you ace the take home test.

Apparently its got something to do with the whole team getting together and publicly code reviewing your assignment. And if they feel like the code doesn't fit even to the exact size per their liking, its a reject.

I have had a similar experience, where a perfectly done assignment with unit and functional tests, documentation and all the bells and whistles, It took me two full sleepless nights to get it done. But apparently they rejected, for me not using a specific design pattern which one reviewer likes.

It's absurd to the point of disbelief.

Trust me the take home assignment is a sham. People inside get their friends and alumni hired with no hassles. They only give you these tests because they need reasons to reject you.

> We then gave the interviewee four days to complete the task with a recommended time of 2 days.

How magnanimous. Who the hell is subjecting themselves to this?

People who are between jobs and whose bank accounts are running perilously low.

Actually, I did one of these jumping from one megacorp to another. It took up my entire weekend, but I landed the interview. I then had to take 2 days off to fly out for the interview. Luckily, I got the job.

You either got very lucky or you're far more effective at pre-screening companies than I am. The last time I spent that kind of time on a test was for a megacorp in the top 50 of the fortune 500.

The process was much the same as you described: multiple day test followed by taking two days off to fly out for the interview. Except I didn't get the job and nobody who interviewed me ever looked at the take home test.

In a government role, I was required to obtain permission for any outside activity, with meaningful penalties. Significant engagements like that for an interview would be in scope.

The interview process being long or whatever or requiring on-site visits is reasonable, but pre-work is probably eliminating good candidates, or attracting people who don't pay attention to their contracts.

I'm interviewing with 6 companies simultaneously and like you they all think they're deserving of free labor.

I'd wager your company must have the worst devs, those who have little self respect and zero other prospects.

Do you not understand that you're requesting free labor from an applicant? Did you forget the purpose isn't to have the sickest challenge ever but to evaluate someone's code/ingenuity?

Methinks you didn't read GP correctly. They took a 2-4 hr task and let the interviewee take 4 days, not including weekends.

2-4 hours is anywhere from $100 to $800 worth of free labor depending on how the dev values their time. The GP was bragging about how generous they were in providing multiple days and weekends to finish it.

How did this become acceptable practice for hiring people? Do any other industries besides tech do this?

So teams should just not vet candidates? An in person interview is also going to be 2-4 hours, plus travel time. Should we stop doing those, too? How exactly do you suggest teams screen candidates?

Designers are hired based on their portfolio, why are devs treated differently? I have a plethora of options on my github as I'm sure most here do. Why is it not acceptable to judge a candidate by their already written code?

Lots of people would see this as unfair, as they don't have anything GitHub. Either for preferring to do something else with their precious free time, or perhaps slaving full days in their closed-source jobs.

I think the only fair option is an in-person interview without homework assignments. Treats everyone the same and doesn't require inordinate time commitment.

Studying for a programming interview is also transferrable, whereas maybe pieces of your weird homework assignment will be applicable in another requested project.

If I want to interview at 6 places that just use onsite interviewing it's easy to figure out how to prepare.

This is a good point. Also many places don’t even allow open source contributions, even in your spare time.

They are asking the interviewee to complete the same task as done by an existing staff member, and compare the results, how is that "free labour"?

I'm not sure where I stand on this sort of thing to be honest, my only experience of anything similar was a pre-interview task to implement a simple 10 entry LRU cache, and after getting the job I was told the only reason it was there was to weed out time wasters with next to zero skill being sent by the recruitment company.

work is work, whether or not it gets used. arguably this is, though

Any industry where hiring candidates with bad fit is extremely expensive will have a long interviewing process. For the candidate there is not much difference between spending 2-4 hours at home at a time of their choosing and 2-4 hours in a first of the N rounds of on-site interviews. Except they will have to expense transport and work around their job schedule.

Many people prefer this to an in-person whiteboard challenge. It's lower stress, and doesn't require taking additional time off from your current employer.

A 2-4 hour challenge is less time than I've spent in interviews at some companies, and would gladly have done it prior to an in-person interview given the option. That way, the actual interview time could have been spent talking about things of value, rather than having them stare at me while I white-board out problems around graph data structures and recursion and whatever other trite trivia they can come up with.

To be useful for discriminating between competent candidates, a challenge has to either be hard to complete (pass based on completion) or hard to complete to a high standard of quality (pass based on completion+quality). In other words, it has to be at the limits of what's possible for people you expect to apply for the job.

If you give me a 2 hour challenge with a timer so I can't possibly take more than 2 hours, I'm fine with that - but you won't accomplish the "lower stress" goal.

But if you give me a 2 hour challenge without timing? I know for sure other people will have spent 8 hours making super-polished, gold-plated solutions. Can I afford to fall behind the competition? If not, I should spend 8 hours too.

> To be useful for discriminating between competent candidates,

I helped redesign the code challenge at one of the companies I worked for. We (the people across the different technical disciplines we hired for) put a lot of effort into ensuring that:

* the challenge could be completed in 4 hours * the challenge resembled the sort of work people would do on the job * the goal of the challenge was to inform a follow up interview, not punish people * candidates were explicitly told that they were not being timed, but that we expected the challenge should take them up to 4 hours to give an idea of the upper-bound level of effort expected

The single biggest challenge was balancing "what do we need to know" versus "what would we like to know" versus the amount of time we were asking of our candidates.

We didn't hard-fail anyone unless they were blatantly not a good fit from their submission, and I habitually wrote multiple pages of feedback for them so they at least got something for their time if they didn't get a follow up interview.

Again, the purpose of the challenge was not a screen, so much as it was to help guide the in-person interview and so we had something concrete that we could discuss.

Follow up edit: Designing code challenges is hard. You have to avoid something so objective it can be copy-pasted from stack overflow, but objective enough that it can be graded free from personal bias (as much as possible anyway). On top of that, you can't test for MVC+CQRS+FP+SOLID+every-other-possible-thing under the sun, because you have to be respectful of candidate's times.

The previous code challenge (the one I replaced) would frequently take people 8-20 hours to complete, and it was actually pretty simple. As you rightly pointed out though, candidates often read far more into the requirements than were actually there, and would often over-achieve in an effort to stand out. Not only would those submissions waste their time, it wasted ours as well as they took longer to develop feedback from.

I think you're missing the point from the previous poster about (perceived) competition.

Unless your challenge literally stops at, "make X appear onscreen" with no regard for quality, testing, etc. Giving unchecked/unverified time restraints isn't fair. It doesn't matter you're giving more time than it should take to complete. If the task can be done in 2 hours, but you give "6" and Candidate A does it in 3, but Candidate B does it in 32 (but tells you 6) you're ranking two totally different submissions. Candidate B might have a super polished submission, while Candidate A has a baseline submission.

The poster you replied to was suggesting that tests should be either 1) not based on quality of submission and simply rely on difficulty so that only a few candidates can complete them or 2) based on quality and difficulty, but with a checked and verified time to keep the playing field level. However those options are both at odds with "low stress."

Again, in our case, we didn't use it as a screen unless the code was an unmitigated disaster or didn't complete the basic requirements.

So, if someone submitted something that could have been done better, see might ask what could be done to improve it in the follow up interview. A good answer would include the technical bits, and a better answer would describe the time recommendation and why a more polished version would take longer or be considered over-architecting.

This purpose- to inform a conversation with a concrete, familiar code base- is better than both take-home screen assignments as well as the in-person whiteboard exercises, which encourage route memorization and don't reflect the nature of the job we were hiring for.

Are you at liberty to elaborate what the discarded longer challenge was?

Every "take home" challenge I've ever been presented with happens before the onsite interview and doesn't replace it.

Right. Some companies use it as a screen, we used it to inform the interview.

Two of the last three companies I interviewed at had at least 4 hours of in-person interviews. One of those (for a regular developer position, around 7 years ago) had multiple whiteboard sessions with different people. The problems were trivial to solve if you had access to google or had memorized basic data structures and related algorithms. Instead of taking half an hour or an hour talking about the code I'd written to solve a problem representative of what the actual job entailed, we spent 4x as much time (during business hours) talking about things freshmen learn in CS programs (I'm assuming here, I didn't get a CS degree).

Even if the total amount of time is the same, or even greater, I still prefer the take-home assignment because I can do it on my own time, rather than taking off of work, and - if done correctly - makes for a much interview.

As a candidate, you learn far more about the people you're going to be working with than in a whiteboard session as well.

What surprises me the most is people making 6 figure salaries complaining about making 1k investments into themselves.

Absolutely worth it if it gets you the right job. Hell, most people pay multiples of that through recruiters and referrals.

If you're on $130k and take two days of leave for interviews, you're already making a $1k investment.

How many jobs do you apply to when you're searching? More than a few I'm sure.

Maybe 10% of them get back to you with an interview request and that leads to a code challenge. It's highly plausible for a senior Dev to be simultaneously interviewing with multiple companies and have 5-10 pending code challenges. That shit adds up...

It also costs the company thousands of dollars in opportunity cost to interview 10 peolpe full days to hire 1.

But the company sees the value in doing it, and many people dont see the value in doing it for themselves.

>the purpose isn't to have the sickest challenge ever but to evaluate someone's code/ingenuity?

This rings so true for me re: the last shop I was at. People got off giving "hard" problems and watching dev suffer and then giving high praise for the ones that could take the heat. That place was really toxic.

We give a test that should take 1-2 hours, and give them an entire 7 days to do it. I feel like you might be being unnecessarily restrictive in your time limits.

What I've seen is that people spend the time they'll spend on it, and that's it. Only 1 or 2 people over the last few years have said they ran out of time, and the quality of their code showed that they weren't a fit for the company anyhow.

Reply on news.ycombinator.com