HNHacker News
TopNewBestAskShowJobs

gaylemcd

550 karma · joined March 24, 2013

http://www.technologywoman.com http://www.careercup.com
submissionscomments
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
> If you just recite problems from Cracking the Coding Interview, you have pretty limited insight into the candidate's abilities.

I agree with this. I tell companies to not ask questions from Cracking the Coding Interview. But that doesn't mean those style of interviews are fundamentally broken. It's just stupid to ask questions that candidates are likely to know.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
No, it is related to diversity. I agree that it's unfair to expect all candidates to work on a side project. Some companies expect this out of all candidates (which is bad). Others see it as a plus (probably okay).
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
I actually agree. This is probably all it was.

But the other interviewer still handled it appropriately.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Happy to give you my thoughts, but I'm not sure what you're suggesting.

I do agree that lots of companies create bad interview processes -- for coding interviews and other interviews. Those issues should be addressed.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Lots of people get offers with little to no prep.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
So what do you suggest?
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
And? By your logic, if a lawyer tells you how to handle some unfair legal situation, that lawyer is doing something wrong. Because they're profiting from something that's unfair.

When I talk to a company about how to create a hiring process, I tell them that it's unfair to expect side projects from candidates.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Do you understand that there is a distinction between saying "this is how many companies think" and "this is how companies should think"?

I agree that companies shouldn't demand side projects. However, if you're having trouble getting noticed, it can help.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Oh, you don't have to have side projects. But if you are struggling to get noticed, it can help.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Do you truly believe that all those top tech companies have decided on this process to boost the egos of the interviewers? Are you saying this because you have actual evidence? Have you talked to interviewers who admitted this was their personal motivation?

I assure you that the purpose of these questions is to identify good engineers. You might think it fails to do so, but that is the purpose.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
So if whiteboard interviews are stupid and ineffective, what process do you think is better?

I assure that the big tech companies absolutely do care, but they struggle to find something better.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
They have lots of impetus to change. It's just that they struggle to find a better system.

Talking about prior experience? Flawed for the reasons you mentioned.

Take home stuff? Flawed -- cheating and other issues (I discuss this in another comment).

Work with us for a week? Doesn't scale. Unfair to candidates. Lots of issues here.

Sit down and help me fix this bug? Flawed:

1. Huge bias based on whether they know this particular skill set -- the right tools, etc.

2. Pretty arbitrary as to whether they find that particular bug.

3. It can't be a real bug. It has to be a toy project in order to get a consistent evaluation.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
1. I'm not saying a week for preparation overall. I'm saying a week to learn the core algorithms and data structures. Frankly, that's probably much less than a week to learn.

2. Yes, it does throw false negatives. Lots of people get hired with zero prep though. I generally say about a month of prep is where you hit pretty low returns from more time. But that doesn't mean that you need a month, or that a month will be enough for you.

3. I'm not sure I see your point here. If coding interview prep is a different skillset from coding competitions, then you wouldn't do coding competitions in order to do interview preps.

4. Some candidates do shut down because it's not what they're used to. A well-trained interviewer can mitigate this to some extent (not always). Some people will also shut down at a computer. Interviews will always be an artificial environment and people can freeze for different reasons. At a whiteboard, it's often a bit easier to unfreeze candidates and create a happier atmosphere.

5. That's fine. As long as I can read it, it's fine. And if I can't read it and you can explain to me what you're writing, that will also work.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Yes and no.

Dressing abnormally does indeed show that you're either unfamiliar with the norms in the industry or you're deliberately violating them. Agreed.

In finance, it's sort of reasonable to be concerned about someone showing up in jeans and a hoodie. A good part of that job is being able be "corporate" and adhere to norms. If you can't do that in an interview, how will you treat clients?

In a coding interview for a tech company, adherence to social norms is not particularly valued. Weirdness is tolerated, if not celebrated. So it's fairly different.

Tech companies would be far more accepting of a candidate in a suit than a finance firm would be of a candidate in jeans and hoodie. That said... it's still probably best to just avoid the risk of subconscious (or even unfair conscious) bias.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Yeah, there are interviewers who run crappy interviews. This is not a flaw with the process though. If you replace it with any other process, you're going to find interviewers who are bad at evaluating those too.

For example, on behavioral questions, I've seen many interviewers who think "In our company, we handle disagreements by doing X. So I'm going to see if, when the candidate hit a disagreement, they handled it by doing this."

But that's not right. Different companies have different cultures. What's good in one company is bad in another. This is essentially the behavioral question equivalent of the scenario you're raising for technical interviews -- people looking for particular answers, and not considering the context.

Both situations definitely happen, and are very common. But that doesn't make the process flawed. It generally means that the company hasn't adequately trained interviewers.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
I don't think that's the case... are you sure you just didn't hit the "read more" button?
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
I think it's well intentioned but highly flawed.

1. The time commitment issue.

2. It allows a company to give a bunch of "interviews" (assignments) even for candidates they aren't serious about. It can waste a lot of candidate time.

3. There is so much cheating on these that you really can't use it for evaluation. It can really only be a screening tool (which makes 4+ hour assignments really unfair).

4. It primarily evaluates current skillset, not how good they'll be with a bit of training. It's the latter that you care about.

5. They're generally given with little thought about what the company really values. It's typically "uh, gee, we do iOS, give them an iOS app." These projects generally aren't very good at narrowing in on very specific things, like problem solving skills.

6. You don't get a lot of context about why the candidate did things a certain way, and how they would have done things different with some guidance. 7. The results vary dramatically based on how much time a candidate spent on something.

... among other issues.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
OP here. I do absolutely realize that candidates getting rejected and then getting an offer later is... well, not quite the "definition", but it's a sign of a flaws in a process. However, you don't scrap the buggy program if there's not a better replacement; you fix the program. And you don't scrap this interview process if the alternatives are even worse.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
No, I don't publish times. The times are misleading. Interviewers often give hints, which can dramatically change the amount of time. And there can be random factors that change the amount of time.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Basically, yes, do more.

If you want to flip a memoization problem into bottom-up dynamic programming: 1. Make sure you really understand the memoization approach 2. Look at the base cases. What are the very last things the recursive approach does? 3. Build up the next case from the base case. 4. Repeat

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
That is really, really not true. An unintelligent engineer will never be able to practice enough to do well on a well-conducted interview at Google. (They could in rare cases do well if all their interviewers pick common problem.)
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
When I do interview training, I strongly discourage memoization / dynamic programming question. It's not that you have to have practiced this to do well. It's that you can have an okay candidate who practices it and then does well.

A properly conducted technical interview will not be about having seen the algorithm before. Some companies screw it up though. But then, these same companies also screw up other parts of the interview process.

I disagree that using certain languages makes a lot of problems trivial. Maybe it makes certain problem trivial, but those tend to be easier ones anyway. They're already trivial. And even if your language has a function that performs the exact thing being asked, the interviewer can easily ask you to go implement that thing.

Java is fine for a whiteboard. So is python. So are most languages.

No, python has not become synonymous with pseudocode. I'm not sure what you mean by that.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
That's not necessarily the case. People who are in shape have longer lifespans than those who aren't (probably true; let's assume it is). But if you look at just people who complete a marathon in under 3 hours, you might not see a correlation between their speed and their eventual lifespan. At some point, people might be "good enough" where beyond that is rapidly diminishing returns.

To complicate this: 1. The range in interview performance of those who get hired is really quite narrow. So you're looking at essentially the difference between a 3.1 and a 3.3. You'll need a lot of data to try to find a correlation there.

2. The interview score is only a partial measure of interview performance. If you and I experience an identical interview, I might call that a 3.1 and you might call that a 3.3. The hiring committee will hopefully understand that a 3.1 from me and a 3.3 from you are equally good (because they see our past scores). This makes the data on interview performance harder to correlate, and you'll need even more data. (Or you'll need to properly take this into account in the study, by looking at a score relative to that interviewer's past scores, rather than the score itself.)

3. The above point, repeated for job performance.

4. When a candidate with a lower score gets hired, they're more likely to have something that offset these lower scores. The data set of people who got hired is {people who interviewed well and had good or bad resumes} + {people who interviewed just okay but had great resumes}. When you correlate interview performance of those who got hired, the lower scores in there are skewed towards people with great resumes (or referrals or something else). That factor could totally eliminate any correlation between interview scores and job performance.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
Nope.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
I'm pretty sure I interpreted the comment correctly...
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
>> >> You should know what a binary search tree. You probably don't need to know the specifics of how to balance one. >> Fine. I know this. Will this give me a job? No

No, it's not about knowledge. The knowledge is needed. But the real focus of the interview is to tackle a new question.

>> I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.

The ability to see a hard problem, develop your own solution, and translate that into code is a pretty important skill. "Skills" aren't all about knowledge.

>> If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?

It's a lot more complex than that. It's not just about filtering out bullshitters. It's also about identifying people who might be great but are unable to talk about it. There's also lots of people who are great but haven't been in situations where they've been able to tackle interesting challenges (either because of their inexperience, industry, etc). Or maybe they've done interesting things but in a related field that's hard to understand.

But, yes, absolutely train interviewers. (Interviewing isn't really a role that you hire for.) But even a very well trained interviewer might not be very effective in a behavioral interview.

>> I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?

There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.

>> The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.

The interviewer is comparing the candidate to other candidates on the same question. So if a candidate is getting rejected because they failed to be perfect, this means that other candidates are actually getting to perfect solutions. Or you just have a crappy interviewer who rejects everyone.

>> Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?

If that's the case? Nothing. But that's not the case.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
When people don't get an offer after a technical interview, it's typically about technical skills. Even if they think they did well because they verified their answers online. Often, although their answers were eventually correct, it took them too long to get there which the interviewer perceived as weak problem solving skills.
gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
This is actually a really interesting point.

(Context: In chess, a queen can move along a row, column, or diagonal to attack. The N Queens problem is to place N queens on an NxN chessboard such that no two queens can attack each other.)

No one does poorly here because they are "bad at chess algorithms." They might do poorly because they think they're bad at chess algorithms. But this is not a "chess algorithm." It's an algorithm, with a chess skin.

But, sure, if you're bad at chess algorithms, I'll give you this problem: Given an N by N boolean matrix (all falses), set N cells to true such that no two rows, columns, or diagonals have two trues.

Same question, but with a boolean skin. Now will you be able to tackle it?

So, advice: Don't assume you're bad at __ type of algorithm question. For the most part, this isn't true. At most, there's a tiny bit of knowledge to tackle it (so you're not bad at it; you just don't know something). Nearly every time I hear someone say they're bad at some type of question, it's actually just an insecurity. They aren't even missing any knowledge.

The only partial exception here is recursion/dynamic programming, which does have its own little approach.

At a higher level: Can you ask for a new question if you're bad at that type?

You could, but it's risky. If I ask you a question that really involves pointers and you don't understand them, then okay. But realize that this might be a deal breaker for me. I might need that knowledge, or I might be concerned about the tendency to give up.

You're probably better off just voicing something like: "To be honest, I haven't worked much with pointers. I'm happy to give it a shot though, unless you want to move onto a different question."

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
1. No, you don't need to know everything. You should know what a binary search tree. You probably don't need to know the specifics of how to balance one. (If a company is requiring this knowledge, then they're doing whiteboard interviews poorly [or they have a very specific skillset needed]. That's not a problem with whiteboard interviews. That's a problem with that company or interviewer. It takes maybe a week for a smart developer without a CS degree to learn what they need to know for interviews.

2. Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?

3. Sure, you could do coding competitions if you want. That'll replace existing interview studying; you wrote your comment as though that's in addition to your previous studying.

4. Yes, true, you don't code on a whiteboard. But this, in and of itself, doesn't matter. What matters is whether whiteboards are more predictive and whether they create a better candidate experience. So, forget whether it's "real world" or not. Whiteboards have their pros and cons. The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.

5. You do not need to work on your handwriting. I've done A LOT of interviews and this has almost never been an issue.

gaylemcd··on What I've learned from 100s of interviews with candidates at top tech companies
It's very difficult to correlate interview performance and job performance. No one is willing to hire at random to really get the data. Companies only look at who got hired.

I've seen some companies decide "well, whiteboard interviews are flawed" and just scrap the whole thing and replace it with something else... that's much more flawed. They're really not being honest with themselves about what the flaws exactly were in whiteboard interviews and if those are fixable, or what the flaws are in their new process.

There's a lot that can be done to make whiteboard coding/algorithms interviews less flawed. - Be clear with candidates and interviewers about expectations. What knowledge is okay to assume? What knowledge is not? - Clear guidelines on what makes a good question. A lot of companies ask very simple or well-known questions but think they're emulating what Google does. They are not. - Good interview training. Most companies don't have interview training, beyond maybe basic legal stuff. Even the companies that do have training typically have crappy training. - Interviewers should candidates with certain things that might not be obvious. For example, writing examples of input and output on the whiteboard is helpful. Don't rush into coding.

There is A LOT that can be done to make whiteboard interviews more effective.

You can also replace them with some other things, but understand that those will come with their own flaws. Scrapping whiteboard interviews is not going to deliver you some perfect system.

← PreviousPage 2 of 5Next →