Hacker News Reader

Comment by lisper | Hacker News Reader

I think Triplebyte has the right idea: essentially a take-home test conducted over a period of a couple of days. It allows a candidate to show off what they can do under realistic conditions, instead of allowing an interviewer to poke at what they can't do under highly unrealistic high-pressure conditions. A typical tech interview is more like a spelling bee than a realistic test of a candidate's abilities: if you happen to get a word/question you know, you look awesome. If you don't, you don't. (Not long ago I screwed up a tech interview because I couldn't remember/figure-out-on-the-spot the iteration condition for estimating square roots by the Newton-Raphson method, and I was not willing to cheat by looking it up on Wikipedia while I was on the phone. Their loss.)

I just wish Triplebyte would expand their outreach beyond YC companies.

View on HN

Not long ago I screwed up a tech interview because I couldn't remember/figure-out-on-the-spot the iteration condition for estimating square roots by the Newton-Raphson method

(1) Better to say you were unable to "remember", rather than a "figure out". No one ever "figures out" things like the Newton-Raphson method over the phone -- not even people like Isaac Newton or Joseph Raphson (substituting whatever comparable level of distraction on had to contend with in those days, absent telephones -- "while ordering a beer at the pub", I guess).

(2) The use of this question as a binary hiring filter (mindlessly copy-and-pasted from their rough impression of what ever other company is doing in 2016) was a failure on their side, not yours.

Actually, under non-brain-wedgie conditions, I can re-derive Newton-Raphson because I do have a good conceptual grasp of it. (In fact, after I got off the phone, I did it just to convince myself that I could. It took me about five minutes, which is nothing under realistic conditions, but a long time when you're on the phone.) But at the time I had a brain wedgie, and the interviewer just kept pushing and prodding and wouldn't let it go. It was damned annoying. I was sitting there thinking, why are you so fixated on this?

It was definitely their loss because the technical problems they were facing were very similar to ones that I had faced (and solved!) in a previous startup (network optimization problems). But I never even got to talk about that.

Because of that experience, I now make it point whenever I interview someone to always ask, particularly if they're floundering, "What should I ask you about that will let you show off your strengths?" I don't have any qualms rejecting someone who can't answer that question (and a surprising number of candidates can't).

figure out in the sense of discover? sure

Figure out in the sense of work out the updates, particularly for f:R -> R, if you already understand NR? That's totally doable in 5 minutes with paper; it's just figuring out an x intercept. And remembering/finding f such that f(sqrt(2)) = 0

I've also been asked, in person, the derivative of x^x. Hope you remember logarithmic differentiation! It's not like, had you happened to not have used this trick for a derivative in the last 10+ years, you could look up and recall in 5 minutes given you understand chain rule. I remembered it, but what kind of filter is this?

"Discover" in the sense of "derive ab initio, truly never having seen an iterative root-finding method, or anything analogous to it" -- like Newton and Raphson both did, separately -- I highly doubt it.

Re-derive your (or my) perhaps-more-than-a-bit-stale-by-this-point derivation, from way back when, under non-interview conditions? Yeah sure -- assuming we were actual math majors, or among the 10% or of CS majors who are burned-in math types.

But under interview conditions? Unless the role explicitly requires an actual math background (or a CS background with emphasis in numerical algorithms), really quite a silly thing to expect of someone.

oops -- when I said "figure out in the sense of discover? Sure" I was unclear, but I agree -- that request would be ludicrous.

That said, I have full faith that Gauss would pass that interview. Probably Euler, Tao, Dantzig, and Turing too =P

But it's not entirely unreasonable for optimization/OR/ML people. Mostly they have to remember how to construct the right function to find the zero of. Most applications of NR just want to find the zeroes of gradients of loss functions.

I certainly am not disputing the silliness... I personally am not good at math at a whiteboard while someone is staring at me. And full disclosure: I remember NR from the picture -- the x intercept of the tangent is (hopefully) closer to the zero than the initial guess.

If these companies would just say, in their job descriptions, that that's the calibre of people they need to debug their glorified hotel booking website (or whatever), then that would make things much simpler for everyone:

"When we say we're looking for the next Norvig or Knuth for this role, we mean it -- and you'll be tested accordingly!"

How could you debug a crud app if you're not Knuth?

Makes no sense.

Not to sound too snobby (I would have notrecognized the Newton-Raphson method is by its name either), but I think it is at least plausible to figure out the method on the spot.

If you either know how the first order of a Taylor series looks like or just brainstorm a bit about how the derivative could help you in finding a root, you will figure out the Newton-Raphson method rather quickly. But then, it really depends on the job whether this is actually knowledge the interviewer would want to test.

That said, I completely understand that even if you would have the required knowledge, it is really easy to fumble during a high-stress situation.

I have tried using take-home tests before. The level of plagiarism was astonishing. And that's just the plagiarism I could detect.

Even with a problem that's unique to my organization, I don't know how I could trust that the actual candidate themselves was the one who completed it, and that they completed it without unfair assistance (e.g. from other people). Similarly, I have a habit of looking over someone's resume and picking a few random technologies they mentioned to discuss. I'm surprised how often candidates claim experience with a technology and can't really describe what they've done with it or discuss it intelligently. The level of dishonesty is high.

For all of the downsides of interview-based hiring, at least I know what I'm getting (modulo error bars on assessment efficacy).

> I couldn't remember/figure-out-on-the-spot the iteration condition for estimating square roots by the Newton-Raphson method, and I was not willing to cheat by looking it up on Wikipedia while I was on the phone

I don't ask the kind of questions that require obscure knowledge. I just ask questions that you can problem-solve with regular rational thinking, with questions that usually admit a "naive" brute-force style solution that can be improved upon. If you did want to look up some algorithm that would be fine with me, but I am more impressed by a quick and confident reasoning through the problem followed by a fluid implementation of the naive solution, than I am of sophisticated solutions with better algorithmic performance or accuracy.

Isn't getting back plagiarized answers a good indication though? Did you tell them you wanted it to be their own work?

As I have gotten older one of the things I'm cognizant of is the difference between what I mean and what my kids hear. So "take this problem home and solve it" from me means that they should take it home work on it themselves and bring back their best solution. But they might hear it "take this out of my sight, put the answer on it, and bring it back to me." where in the land of "out of sight" there are really no restrictions on what or how they get it done. As a result I find myself being a bit prescriptive, saying "Take this home and work on it, and bring me your solution tomorrow, we're going to talk about your process of how you got to the answer. When we talk, saying 'I asked on Stack Overflow' isn't going to be a good answer, ok?"

Their response will be informative, from moral outrage that I would suggest they can't do the work, to understanding that the integrity of the process was part of the problem set. All of that helps you understand how they approach being asked to do things.

> plagiarism

Triplebyte controls for that by having a followup on-line session where you have to walk them through your code and modify it on the spot. They probably use regular plagiarism filters too.

>Similarly, I have a habit of looking over someone's resume and picking a few random technologies they mentioned to discuss. I'm surprised how often candidates claim experience with a technology and can't really describe what they've done with it or discuss it intelligently

I know what you're saying, but there's different levels.

Several years ago, I read a couple of books on Hibernate/JPA and played around with it on some toy projects. Where I work never really ended up using it.

Should I put Hibernate on my resume? I know a lot more than someone who has never used it, but it's been a few years since I've done anything with it, and I never really used it in production/anger.

I wouldn't pretend to be an expert (and I'm guessing some people with my level of experience would), but I still think it's OK to put it on my resume (although I'm not sure if it's on my current resume because I don't really care to work with it :)).

I solve this by not using a skills block or anything like that. Partially because I think they're stupid but also partially because IMO it's a no-win situation to figure out what it implies, you'll always mismatch with some interviewer.

We give take home problems and have no issue with plagiarism. In fact we actively say use the internet, books, etc..., and we still get solutions that are wrong or do not even compile.

The ones that do come back and look good, give us a starting point for the interview. If the person did just copy it from somewhere else they have will have a hard time talking about the solution and various pros/cons.

This is how we do things as well. When I walk around our office I always see people looking at SO, Google and others. Why would I hold a candidate to a different standard?

At one company that did a take home, the first step of the onsite was some pair programming to add a few new small features to my submitted solution.

> I'm surprised how often candidates claim experience with a technology and can't really describe what they've done with it or discuss it intelligently. The level of dishonesty is high.

This is a requirement to get past the HR gatekeepers at most companies. That may not be the case at your company, but how can I tell? When a critical mass of companies require you to lie before you can even talk to a technical person, then you're going to get lied to as well.

I don't have time for that. Nobody wants to spend their evenings working on exams. I'd prefer a few hours of onsites any day.

It's more reasonable for TripleByte because they share across companies. I agree, for a single company I have to be really interested to agree to give up 4+ hours...

"Not long ago I screwed up a tech interview because I couldn't remember/figure-out-on-the-spot the iteration condition for estimating square roots by the Newton-Raphson method, and I was not willing to cheat by looking it up on Wikipedia while I was on the phone. Their loss."

Out of curiosity, what type of developer position were you interviewing for?

Not quite sure how to answer that. Do you mean what was the label on the job req? I don't remember. I think it was "Senior software developer" or something like that. Is that what you meant? (And if so, why do you ask?)

I'm trying to figure out what software development job that would ever come up? graphics?

Ah. No, definitely not graphics. I can't say much without giving away the identity of the company, but like I said, the technical side of their business had to do with network optimization. It was "Uber for X" for some value of X.

> what type of developer position were you interviewing for?

A square-root estimator developer position, of course.

I could see this question being a valid question to ask for a research scientist role if the applicant has a PhD in CS/Mathematics/Operations Research and their research area is optimization or machine learning, or they claim to have taken coursework in convex optimization. Because you definitely have heard of Newton's method in those contexts.

There was no take home test for me. It was a 45 minute multiple choice test, a timed coding problem after that which you could skip if you did well on the test, and then a 3 hour or so chat that involved an hour of coding something and various systems and algorithms questions

Reply on news.ycombinator.com