I agree with this. I tell companies to not ask questions from Cracking the Coding Interview. But that doesn't mean those style of interviews are fundamentally broken. It's just stupid to ask questions that candidates are likely to know.
550 karma · joined March 24, 2013
I agree with this. I tell companies to not ask questions from Cracking the Coding Interview. But that doesn't mean those style of interviews are fundamentally broken. It's just stupid to ask questions that candidates are likely to know.
But the other interviewer still handled it appropriately.
I do agree that lots of companies create bad interview processes -- for coding interviews and other interviews. Those issues should be addressed.
When I talk to a company about how to create a hiring process, I tell them that it's unfair to expect side projects from candidates.
I agree that companies shouldn't demand side projects. However, if you're having trouble getting noticed, it can help.
I assure you that the purpose of these questions is to identify good engineers. You might think it fails to do so, but that is the purpose.
I assure that the big tech companies absolutely do care, but they struggle to find something better.
Talking about prior experience? Flawed for the reasons you mentioned.
Take home stuff? Flawed -- cheating and other issues (I discuss this in another comment).
Work with us for a week? Doesn't scale. Unfair to candidates. Lots of issues here.
Sit down and help me fix this bug? Flawed:
1. Huge bias based on whether they know this particular skill set -- the right tools, etc.
2. Pretty arbitrary as to whether they find that particular bug.
3. It can't be a real bug. It has to be a toy project in order to get a consistent evaluation.
2. Yes, it does throw false negatives. Lots of people get hired with zero prep though. I generally say about a month of prep is where you hit pretty low returns from more time. But that doesn't mean that you need a month, or that a month will be enough for you.
3. I'm not sure I see your point here. If coding interview prep is a different skillset from coding competitions, then you wouldn't do coding competitions in order to do interview preps.
4. Some candidates do shut down because it's not what they're used to. A well-trained interviewer can mitigate this to some extent (not always). Some people will also shut down at a computer. Interviews will always be an artificial environment and people can freeze for different reasons. At a whiteboard, it's often a bit easier to unfreeze candidates and create a happier atmosphere.
5. That's fine. As long as I can read it, it's fine. And if I can't read it and you can explain to me what you're writing, that will also work.
Dressing abnormally does indeed show that you're either unfamiliar with the norms in the industry or you're deliberately violating them. Agreed.
In finance, it's sort of reasonable to be concerned about someone showing up in jeans and a hoodie. A good part of that job is being able be "corporate" and adhere to norms. If you can't do that in an interview, how will you treat clients?
In a coding interview for a tech company, adherence to social norms is not particularly valued. Weirdness is tolerated, if not celebrated. So it's fairly different.
Tech companies would be far more accepting of a candidate in a suit than a finance firm would be of a candidate in jeans and hoodie. That said... it's still probably best to just avoid the risk of subconscious (or even unfair conscious) bias.
For example, on behavioral questions, I've seen many interviewers who think "In our company, we handle disagreements by doing X. So I'm going to see if, when the candidate hit a disagreement, they handled it by doing this."
But that's not right. Different companies have different cultures. What's good in one company is bad in another. This is essentially the behavioral question equivalent of the scenario you're raising for technical interviews -- people looking for particular answers, and not considering the context.
Both situations definitely happen, and are very common. But that doesn't make the process flawed. It generally means that the company hasn't adequately trained interviewers.
1. The time commitment issue.
2. It allows a company to give a bunch of "interviews" (assignments) even for candidates they aren't serious about. It can waste a lot of candidate time.
3. There is so much cheating on these that you really can't use it for evaluation. It can really only be a screening tool (which makes 4+ hour assignments really unfair).
4. It primarily evaluates current skillset, not how good they'll be with a bit of training. It's the latter that you care about.
5. They're generally given with little thought about what the company really values. It's typically "uh, gee, we do iOS, give them an iOS app." These projects generally aren't very good at narrowing in on very specific things, like problem solving skills.
6. You don't get a lot of context about why the candidate did things a certain way, and how they would have done things different with some guidance. 7. The results vary dramatically based on how much time a candidate spent on something.
... among other issues.
If you want to flip a memoization problem into bottom-up dynamic programming: 1. Make sure you really understand the memoization approach 2. Look at the base cases. What are the very last things the recursive approach does? 3. Build up the next case from the base case. 4. Repeat
A properly conducted technical interview will not be about having seen the algorithm before. Some companies screw it up though. But then, these same companies also screw up other parts of the interview process.
I disagree that using certain languages makes a lot of problems trivial. Maybe it makes certain problem trivial, but those tend to be easier ones anyway. They're already trivial. And even if your language has a function that performs the exact thing being asked, the interviewer can easily ask you to go implement that thing.
Java is fine for a whiteboard. So is python. So are most languages.
No, python has not become synonymous with pseudocode. I'm not sure what you mean by that.
To complicate this: 1. The range in interview performance of those who get hired is really quite narrow. So you're looking at essentially the difference between a 3.1 and a 3.3. You'll need a lot of data to try to find a correlation there.
2. The interview score is only a partial measure of interview performance. If you and I experience an identical interview, I might call that a 3.1 and you might call that a 3.3. The hiring committee will hopefully understand that a 3.1 from me and a 3.3 from you are equally good (because they see our past scores). This makes the data on interview performance harder to correlate, and you'll need even more data. (Or you'll need to properly take this into account in the study, by looking at a score relative to that interviewer's past scores, rather than the score itself.)
3. The above point, repeated for job performance.
4. When a candidate with a lower score gets hired, they're more likely to have something that offset these lower scores. The data set of people who got hired is {people who interviewed well and had good or bad resumes} + {people who interviewed just okay but had great resumes}. When you correlate interview performance of those who got hired, the lower scores in there are skewed towards people with great resumes (or referrals or something else). That factor could totally eliminate any correlation between interview scores and job performance.
No, it's not about knowledge. The knowledge is needed. But the real focus of the interview is to tackle a new question.
>> I want to be hired for the skills I'm bringing in and not for my ability to solve toy problems on a whiteboard.
The ability to see a hard problem, develop your own solution, and translate that into code is a pretty important skill. "Skills" aren't all about knowledge.
>> If the interviewer cannot filter out bullshitters, the interviewer is incapable of the "job of interviewing". Maybe hire better interviewers. Why is the burden on the candidate?
It's a lot more complex than that. It's not just about filtering out bullshitters. It's also about identifying people who might be great but are unable to talk about it. There's also lots of people who are great but haven't been in situations where they've been able to tackle interesting challenges (either because of their inexperience, industry, etc). Or maybe they've done interesting things but in a related field that's hard to understand.
But, yes, absolutely train interviewers. (Interviewing isn't really a role that you hire for.) But even a very well trained interviewer might not be very effective in a behavioral interview.
>> I agree with this, only so long as the interviewers don't fret about some stupid edge case. Why is there an expectation of perfect code in 15 mins on a whiteboard? Who does that in production?
There generally isn't. What an interviewer should be looking at is not some sort of percent correct element. It's about signal. So if a candidate is totally unable to understand an obvious, that might worry me. The signal the candidate is sending is a poor understanding of details. (Or, maybe not. It depends on the situation.) What I train interviewers on is that correctness in and of itself is not relevant. Why a candidate was correct or incorrect might be.
>> The interviewer knows about these edge cases because they've seen the problem before. But I can guarantee that the same interviewer would not be able to 'perfectly' solve some problems that I pose them without having seen them before.
The interviewer is comparing the candidate to other candidates on the same question. So if a candidate is getting rejected because they failed to be perfect, this means that other candidates are actually getting to perfect solutions. Or you just have a crappy interviewer who rejects everyone.
>> Which brings me back to the question. What is the point of the coding interview (or perfection in coding interview) if the only people who can solve them are the ones who have seen the problem before?
If that's the case? Nothing. But that's not the case.
(Context: In chess, a queen can move along a row, column, or diagonal to attack. The N Queens problem is to place N queens on an NxN chessboard such that no two queens can attack each other.)
No one does poorly here because they are "bad at chess algorithms." They might do poorly because they think they're bad at chess algorithms. But this is not a "chess algorithm." It's an algorithm, with a chess skin.
But, sure, if you're bad at chess algorithms, I'll give you this problem: Given an N by N boolean matrix (all falses), set N cells to true such that no two rows, columns, or diagonals have two trues.
Same question, but with a boolean skin. Now will you be able to tackle it?
So, advice: Don't assume you're bad at __ type of algorithm question. For the most part, this isn't true. At most, there's a tiny bit of knowledge to tackle it (so you're not bad at it; you just don't know something). Nearly every time I hear someone say they're bad at some type of question, it's actually just an insecurity. They aren't even missing any knowledge.
The only partial exception here is recursion/dynamic programming, which does have its own little approach.
At a higher level: Can you ask for a new question if you're bad at that type?
You could, but it's risky. If I ask you a question that really involves pointers and you don't understand them, then okay. But realize that this might be a deal breaker for me. I might need that knowledge, or I might be concerned about the tendency to give up.
You're probably better off just voicing something like: "To be honest, I haven't worked much with pointers. I'm happy to give it a shot though, unless you want to move onto a different question."
2. Yes, I understand that interview prep might interfere with other obligations. You don't have to study. But it'll help you. I'm not sure what you expect here. Do you want companies to hire based on how smoothly you're able to bullshit about what you can claim to have done in the past?
3. Sure, you could do coding competitions if you want. That'll replace existing interview studying; you wrote your comment as though that's in addition to your previous studying.
4. Yes, true, you don't code on a whiteboard. But this, in and of itself, doesn't matter. What matters is whether whiteboards are more predictive and whether they create a better candidate experience. So, forget whether it's "real world" or not. Whiteboards have their pros and cons. The major benefit of a whiteboard is that it allows a candidate to not get caught up in details, like the exact syntax for some function (who cares?) or writing in all the details of some trivial thing just so it can compile. It allows us to focus on the big picture. It encourages more thought process and communication. On a computer, many candidates just shut down -- they try to get things out as fast as possible, and they stop thinking about what they're actually doing.
5. You do not need to work on your handwriting. I've done A LOT of interviews and this has almost never been an issue.
I've seen some companies decide "well, whiteboard interviews are flawed" and just scrap the whole thing and replace it with something else... that's much more flawed. They're really not being honest with themselves about what the flaws exactly were in whiteboard interviews and if those are fixable, or what the flaws are in their new process.
There's a lot that can be done to make whiteboard coding/algorithms interviews less flawed. - Be clear with candidates and interviewers about expectations. What knowledge is okay to assume? What knowledge is not? - Clear guidelines on what makes a good question. A lot of companies ask very simple or well-known questions but think they're emulating what Google does. They are not. - Good interview training. Most companies don't have interview training, beyond maybe basic legal stuff. Even the companies that do have training typically have crappy training. - Interviewers should candidates with certain things that might not be obvious. For example, writing examples of input and output on the whiteboard is helpful. Don't rush into coding.
There is A LOT that can be done to make whiteboard interviews more effective.
You can also replace them with some other things, but understand that those will come with their own flaws. Scrapping whiteboard interviews is not going to deliver you some perfect system.