Tech sector job interviews assess anxiety, not software skills: study | Hacker News Reader

Tech sector job interviews assess anxiety, not software skills: study

news.ncsu.edu

1,782 pointssizzle1,141 comments

All: please don't miss that there are multiple pages of comments. The top few subthreads have become so large that they fill out this first page entirely. You have to click 'More' at the bottom to see the rest (there are over 700 at this point).

https://news.ycombinator.com/item?id=23848039&p=2

https://news.ycombinator.com/item?id=23848039&p=3

We're working on performance improvements that will hopefully allow us to go back to HN's original style of one big page per thread (not infinite scroll, don't worry). In the meantime please look for those 'More' links when the total number of comments is over 250 or so.

I conducted a couple hundred interviews for my first FAANG employer, and I was constantly amazed at the percentage of candidates with years of Microsoft or Facebook experience on the resumes who apparently did not know how to program. I always thought, 'huh, guess I know why they quit after 3 years, amazing that they all lasted this long."

Then I interviewed for another company and utterly bombed. It became suddenly clear to me that I had been an idiot. Of course nearly all of those candidates were perfectly good programmers. They had had shit interviewing days, probably mostly due to nerves, but they probably would have mostly been perfectly good employees. How frickin' arrogant I had been for concluding that people who couldn't solve incredibly high stakes algorithm riddles on a whiteboard in 45 minutes with enough speed and flair were somehow not qualified to be my coworkers.

You've been arrogant for years and you know what? So have I. I and many, many people in this industry have been doing this for years.

What's different from you and many other people is that you admit and face your bias rather than justify it.

I'm seriously curious what google interview board members have to say about this study. People like Gayle Laakmaan have been saying things to justify the whole process for years; but now that there's actual science, what do they have to say in the face of science?

From what I have read from Gayle Laakmaan, they know this throws away many good engineers, but it is acceptable to them because it doesn’t let bad ones through.

This is essentially the point of a lot of hiring and performance evaluation theory.

E.g. it's acceptable (but not ideal) not to promote talented commanders. It's catastrophic to promote someone to General who isn't ready or is unsuited.

Software engineers are not commanders or generals and it is NOT difficult to fire someone in the United States of America. Everyone in this industry has a at-will contract.

However, high rates of employees getting fired is bad for culture. People are always anxious about their jobs - if they see a few colleagues fired for nonperformance, a number of them will be driven to anxiety. (I am one of these people, and being anxious about job security ironically makes me much worse at my job)

Right but I also do think that FAANG is over-estimating how many of and how bad these "bad apples" aka. slighly above average developers would be and how terribly difficult all the work they do is.

When you have your pick of the bunch, of course you're going to look at slightly above average as "bad".

I'm not sure how much engineer skill matters though. One slightly above engineer might be fine. But if half your org is made up of slightly above average engineers, I feel like there will be a knock-on effect.

I don't know maybe my view of slightly above average is inflated but I think of that bunch (which I might be a part of) as being easily pulled in by good technical leadership. If only to make their life ultimately easier / less painful. You don't need 60-70% amazing engineers to do that. But I guess they can do whatever they want. They have infinite money and unfortunately a good reputation (in many cases unwarranted). They've become like the Harvard or Yale of tech. Oh you went to Google, great come on in and join us.

One of the things I've liked about freelancing/contracting is that I know I'm going to get "fired" (we call it "successful completion of the project" but its the same thing, just that they would like to have me work for them again). And since it happens on a regular basis, I also am practiced in finding a new job and I know about how long it will take and what types of prospects to look for. It's really reduced job anxiety for me, although I never had too much.

Although as a counterpoint, it's also easier to get work, because part of the value I'm providing is that you "fire" me whenever you want and no hard feelings. So nobody does "leetcode interviews", because it's not like we're getting married, like an employee. So there's much lower risk to hiring someone, because a bad hire doesn't infect anything, because you expected it to be temporary in the first place.

And yet, time and time again we see that Generals promoted during peace-time rarely have the skills necessary to lead men under intense combat conditions. The best generals are identified on the battlefield.

Coming from a military background, I can confidently say that promotions at that level are more the end of a political process than a true assessment of skills...

But they're not aware that the filter is just anxiety.

Google is basically using anxiety to filter good candidates and eliminate false positives which works in a sense but is still highly illogical.

Why not use a technical filter to filter for technical candidates? Anxiety seems like a pointless filter.... how does that even eliminate false positives?

it would seem it might be 'letting in' a whole load of people who don't have appropriate anxiety responses. There are some situations where anxiety is perfectly normal, and perhaps even useful, but if you screen out people who have normal anxiety, what impact does that have on your company operations and culture?

I'm a pretty anxious person, so much so that I've pretty much retired instead of doing anymore technical interviews. I had a discussion with a very anxious, but brilliant software developer a few years back and we came to the conclusion that as long as we don't let our anxiety get too far out of hand it's actually kind of a superpower in the job setting. That's because that niggling anxiety you get in the back of your head as you're coding generally makes you more careful about how you're designing and testing your code. You're more careful about security, safety and correctness. Whenever I get anxious while coding I stop and ask myself if maybe it's a message that I need to listen to - maybe it's trying to tell me to tread carefully because I'm entering an area that's potentially problematic.

People without that niggling sense of anxiety always in the background, the folks who are easily going to ace the programming interview because their anxiety levels are so low, those folks, I theorize, are potentially not going to be so careful. Now you could argue that that's a plus in many situations - a startup that needs to get code out the door right away, for example. But it really depends on the domain. For critical systems in applications like healthcare, avionics or robotic control I think the anxious coder is the one you want.

Companies that completely weed out the anxious programmers, as you imply, will have a different culture with perhaps too much emphasis on risky behavior.

> But they're not aware that the filter is just anxiety.

Could this be deliberate? As working with people with anxiety problems is often a pain in the ass as they won't report problems for fear of seeming stupid or ask for help.

Confident people are far easier to work on a team with.

I've definitely repeated that line in the past to justify tech industry interviewing practices - but sometimes it goes beyond "setting a high bar" to something more toxic: an interviewer using the interview as a chance to show off how smart _they_ are, rather than assess the candidate; deliberately creating high-stakes "pressure cooker" environments because, you know, that's just how we work here and they should get used to it; and so on. These things then get rolled into the same justification: after all, we're an exclusive club of ultra-smart 10x ninja pirate Jedi, which of course means we should have the default assumption that others don't belong in this club.

I wonder now: for every "bad one" not let through by this sort of process, how many other "good ones" look at it and say "no thanks"? How many hear about this sort of hazing and are dissuaded from applying in the first place? How many experience it one too many times and just quit the industry altogether? How many of those "bad ones" are even objectively bad, and not just having a bad day or intimidated by a process rife with both intentional and unintentional hostility? In other words: are we actually assessing what we claim to assess with _any_ predictive accuracy, and is the collateral damage to company reputation and the pool of available candidates - a damage often hidden to the individual companies that impose this process - even worth it?

I think it goes far beyond that. I'd bet dollars to donuts that you will find both protected and unprotected groups over represented in the false negative pile. Do women have more test anxiety then men? If so, guess what your process is selecting for. Do minorities have more self doubt than majorities. Same thing. And so on.

And that doesn't address the file drawer effect. I'm 54. I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation. So I don't go on FAANG type interviews. I know I will fail, and if I don't my offer will be predicated on that apparently lower performance. At best I will have an utterly miserable experience. So I self select myself out, mostly on age.

This is not good, for so many reasons.

> I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation.

This made me chuckle, because it's absolutely how it is.

Some other comments here talk about how nobody creates new algorithms these days unless they are a researcher. But it's not true at all! Like you I'm creating new algorithms and other tricky techniques all the time, but I'm not fresh on the stuff I learned at school... or so I thought.

I have been really surprised to find some screening interviews recently asked me shockingly trivial questions. So simple that I stumbled over myself trying to simplify the answers, thinking "I know too much about this topic and need to keep it simple", and "can these questions really distinguish candidates?".

I did the TripleByte online test recently and found the questions much simpler than I expected, including those in languages I've never seen before. That is not TripleByte's reputation. I see people writing about how difficult they found the questions. (Admission: I didn't score all 5s, but I can't figure out why unless it's timing as I think I answered them all correctly.)

So I'm thinking, perhaps it's not so bad, people just make it sound bad because there's a wide variation in people's knowledge, abilities and expectations.

If you are thinking you might apply for something like a FAANG or other hard-reputation company, and feeling put off by the horror stories, I would say, just take a look at the old stuff a bit for a refresh, then give it a try and you might be pleasantly surprised to find their reputation is because people less skilled than yourself found it hard. Their "hard" might not be hard for you.

You'll probably still fail the interview if it's a FAANG because they are so selective, but I would bet my dollars to donuts that it would be more refreshing than miserable if you're not attached to passing.

I know your point isn't about yourself, it's about bias in hiring, including bias due to perception by the candidates, but I wanted to address that side point about feeling there's no point applying. That sounds like anxiety to me. (I have it too, I'm trying to get over it.)

>And that doesn't address the file drawer effect. I'm 54. I can code a binary tree, hash table or what have you, if I have to, but I am not as practiced as somebody fresh out of school, because that heavy lifting is done by libraries, and I'm busy contributing original mathematical algorithms that no one ever asks about because they don't have the background to understand the explanation.

Ooohhhh this is frustrating. Nothing is more frustrating than telling an interviewer about something cool you've done, and realizing they don't understand, but also don't care.

Yes, I remember one guy asking what I would when a page was going slow. I explained how I would work backwards checking the network tab in the browser, make sure it was the endpoint that was the bottleneck, blah blah blah to the database. He just wanted me to say "use explain" - which i would have got to if he had bothered listening a couple more minutes. Then he wanted me to do a "challenge" that was expected to take around 10 hours. No thanks, you didn't convince me I want to work for you.

Gayle Laakman specifically has a business empire dedicated to help people get through the process. Regardless of the actual merits of the process her opinion is going to be biased.

I don't think it's relevant. Her business is designed to get you to game and pass the interview regardless of whether or not the interview is effective.

Therefore her business interests are orthogonal to her opinions on "interviews." She may still be biased, but I don't think business interests color her bias.

Quite a racket, being part of setting up this gauntlet and then selling books on how to get through it. Some might call that a conflict of interest, but there's no such thing as ethics for the nobility so to hell with what the peasants think.

I'm not a googler (but I've interviewed there) and I disagree somewhat with the original premise. I think whiteboarding does filter out bad candidates but also filters out good candidates who fail the anxiety test. Google, with their massive pipeline, probably doesn't care about the false negatives enough to change; they still find enough people to hire. It's the companies with smaller pipelines that need to scoop up these good candidates.

In general, hiring practices probably optimize for filtering out bad (or at least somewhat speculative) hires over passing on a potentially really good candidate (especially if there are any concerns to go along with the overall positives).

Of course, if the pipeline is massive (whether it's jobs, schools, etc.) this tendency gets amped up even more and anyone who doesn't come across as pretty much perfect on all dimensions--whether they are or not--is going to get dinged.

I used to be reasonably good at algorithmic stuff in university 20 years ago. Since then I hardly ever practice. It's not that I am bad at them, now, but its just so hit and miss whether an answer will come to me in time, you might as well toss a coin. If I was doing algorithmic stuff every day I have no doubt I would be better. Usually what happens is I get out of an interview and a an answer (or a better answer than I gave) pops into my head. Like I say, might as well toss a coin.

It seems you cannot see FAANGs perspective on this

There are other people who always fail those coin tosses

She will just say that the process is designed to minimize false positives and the expense of more false negatives. A bad hire is more expensive than a no hire.

The problem is this isn't what gets you the smartest people on the planet. A small number might be interested in DS&A to that level, but most are not. They will learn enough to know how to google what they need and move on to what they are interested in. Smart people are constantly getting job offers from coworkers and bosses who move to other companies if they don't start their own.

The few smart people who do put in the effort necessary to go to the FAANG companies always leave after getting the golden sticker on their resume. They leave behind them a residue of SAT preppers who have PhD's in inversing binary trees.

> She will just say that the process is designed to minimize false positives and the expense of more false negatives. A bad hire is more expensive than a no hire.

That's what they keep saying. However, it remains to be proven that regurgitating answers on a whiteboard in 45 minutes implies that the candidate is not a false negative, or even a true positive to begin with.

We don't care about reality, merely perceptions. "Well I humiliated the person for an hour in front of a whiteboard" is the new "Nobody gets fired for buying IBM."

Someone who able to immediately understand and write out the solution to 80% of hard DS&A questions is at least able to write code. The question is if this is honestly better than generating your own FizzBuzz. I don't think so. I have heard rumors that Google has studied employee performance predictions based on interview score and found no correlation. I would bet this is an open secret and no one knows what to replace it with that isn't enormously expensive.

> no one knows what to replace it with that isn't enormously expensive

That's the crazy thing about this. FAANG has near-infinite resources, relatively speaking. Google could be A/B testing their interviewing process, experimenting new approaches, experimenting (as the OP study did) with eye-trackers and other tech to try to gain insights on candidate behavior. They have the "engineering for the sake of engineering" and "innovation for the sake of innovation" culture that allows for moonshots and boondoggles. I don't think they're exactly known for following "if it ain't broke, don't fix it", given how many times they kill off products only to recreate them later on, especially their chat apps.

Yet they stick to the traditional whiteboarding method because-- why?

Maybe Google has studied all of those things and found whiteboarding to be the best predictor. The company seems to be doing well, so something must be working with their hiring. They dropped GPA/college requirements so they're clearly open to non-traditional backgrounds.

> The company seems to be doing well, so something must be working with their hiring.

People use this argument often, but is technical competence of engineers- specifically at passing their interview process -the explanation for their economic success? And not market penetration, near-monopoly positions, great UX, etc.?

Not to mention, as we've seen in the high-profile Facebook SDK crashes this past week, share price doesn't necessarily mean product quality.

>Not to mention, as we've seen in the high-profile Facebook SDK crashes this past week, share price doesn't necessarily mean product quality.

These events are noteworthy because of how rare they are for such a large software-based company.

It was the second time it's happened in a matter of months- the first time was in May.

It's also noteworthy because of how destabilizing this was for the immense number of apps that depend on that software.

It means you're someone willing to spend two to three months on your life sucking a little dick for a chance to work at FAANG... the companies so good they've recommended 15 vacuum cleaners after purchasing one. How many hands do they think I actually have.

The answer is sixteen, you'd know that if you'd spent a bit of time practicing your addition interview questions.

Damn it, back to grinding leetcode!

Boy, am I the only one who thinks working at FAANG is pretty great and worth the time practicing the interview?

The compensation is much better. The management is much better. The work life balance is much better.

I moved from SEA. Everything is worse there.

I read this and feel like people have unrealistic standard. FAANG isn't good enough? Working there for a year probably put your wealth of life at 0.1% of the world. That's not good enough?

That's why people join unicorns. All the comp and potential equity upside of FAANG, less bureaucracy, greater urgency, more sexy work or technologies.

I've seen people say it's like a form of IQ test and I agree with that.

Generally speaking, I think more intelligent people will have an easier time learning the patterns in algorithms.

It's not the best filter, but it's easy to see why it's an attractive option for companies.

intelligence is multi dimensional. Even more so at a working environment. I can't believe algorithm intelligence is a relevant skillset for 99% of the projects. Including projects inside google.

I agree, which is why I said it's not the best filter, but I think it's good enough that companies are happy with the people they hire.

From my experience it won't get you the smartest people on the planet, but it will get you people who _think_ they are the smartest people on the planet.

At Google I've worked with some very humble but also super intelligent awesome engineers, people much smarter than I. Definitely the majority of my interactions. But I've also worked with people who clearly took the "Google hires the smartest people" company line, and their own success at it, a little too personally.

I don't think it's a good message to send. Although to be fair I haven't heard internally in a while.

Also I don't know if it's really a gold sticker anymmore. Nor do I think it's true that the smart people leave. Some do, but honestly, there are some damn brilliant people at Google who have found their brilliant corner and produce brilliance there.

Or some people here who are just brilliant at playing Big Company. Unfortunately there's more of that all the time.

I interviewed a lot of people while I was at Google, and sometimes the people were clearly, clearly too anxious to interview. Like one fresh-grad who's hands were shaking. I thought about how much pressure the kid must have been under.

You try to help them relax, but you don't have much time, and the whole thing is just so unnatural.

Other places I've since been at, and interviewed with, give you a laptop, a problem to solve, and some time. You can focus on the problem instead of the situation much better, and I think that really helps. As an interviewer, it also helps keep some distance from the interviewer. It's too easy to try to micromanage the interview and keep the candidate uneasy. But there's a natural "don't interrupt someone when working" instinct that keeps interviewers more distant when the candidate's programming.

Coderpads on video conference, I think, are pretty good when the candidate has a good space to work in. They're in comfortable territory and you can see them by their keystrokes. The interviewer feels like they're getting more accurate data about how the candidate really operates. I just wish it could do keyboard shortcuts better -- emacs users like me hate seeing Ctrl-N bring up a new window.

The issue of IDE/compiler/keyboard/keybindings is also there, but that's easier to work with.

Microsoft interview. I was just graduating college, and was super anxious walking in to my first interview. The interviewer realized that writing on the whiteboard was going to be awkward for the problem he was asking me to solve so he let me just sit down at his computer while he hung out to to the side and watched me work in notepad. I think that was the best way to do that interview, because the second I started writing code my brain flipped in to code-writing mode and everything relaxed. One of the better interviews in my career.

Nowadays that's "favoritism" in an interview and it wouldn't fly. Everyone must pass through the same meatgrinder (that, surprise, fails people other than white men more often).

Over the past few years in tech, I have interacted with and observed (as a fly on the wall) a wide variety of recruiters, hiring managers, etc. from companies as small as a dozen (usually around the time a founder/CEO brings in external "support").

There exists an incredible amount of arrogance and ego among them by-in-large. The best are humble and curious while maintaining clear and direct communication. To be fair, they are often tasked with making important judgement calls sans complete information, though that's the fault of the leadership/culture as to a poor approach.

At the start of my third year as a student, I biked through a record-worthy rainfall to my first interview with a small local games studio. Showed up completely soaked, but sat down relaxed and feeling reasonably confident. (I had done a handful of unity and opengl projects.)

The interview started with a bit of small talk, I chatted with one of the guys about Brood War and Starcraft 2. Later during the technical interview they asked me about the difference between private, protected, and public and said I was the only student they had interviewed who had answered correctly which was wild and honestly stunned me for a moment.

They liked how in one of my examples of scripting something I had written some fun dialog and I mentioned I did writing as a hobby which seemed like a plus. I talked about which classes I enjoyed the most and how they were challenging/interesting.

I did not get a job offer and learned from a friend that I had come off as "depressed and disinterested" (I don't think they realized he was going to relay that info to me...) All I can do is guess that it's a combination from showing up completely drenched and sharing how I enjoyed those "challenging" classes made it seem like I would get quickly bored of scripting...

After that, I bombed an interview for a QA role because I went into it completely unprepared for just how much it would differ from a programming interview.

Point being those first two experiences got in my head and I basically had anxiety over anything job-hunt related for the next 2.5 years.

When I did get a job, I was extremely happy with the interview process (even though I felt I did poorly on it). Here's more or less how it was structured (TL;DR):

  - Office tour/chat with a programmer who had referred me
  - quick 5 minute introduction to senior programmers in charge of the technical interview  
  - 1 hour alone in a meeting room with a laptop and 4 written questions (a generous amount of time)  
  - ~15 minutes reviewing my answers with the senior programmers  
    - importantly, they gave me a chance to talk about my answers and when I got something wrong they would simply state that it had an error and see if I could spot it  
  - 15 minutes on C++/memory/performance/behavior quirks (important stuff in AAA games)  
  - ~30 minutes talking about stuff I had worked on  
    - occasionally they would mention something related I hadn't heard about and explain it while gauging how well I could follow along

Basically, based on my own interview experience/anxiety if I had to choose an interview method, it would be very similar to what I just mentioned. Seeing a familiar face and the introduction followed by the alone time did a lot to ease my anxiety. The time where I felt most comfortable was talking about the stuff I had done because I was very familiar with all of it/it was easy to talk about.

The process seemed less focused on where I had gaps in my knowledge and more focused if I had a decent amount of knowledge in general and if I had the ability to recognize and correct the gaps in my knowledge.

Sorry if it's a bit of an info dump, but it's something I think back to a lot.

I like that you give an example of what kind of interview you do like.

I've trained for interviewing twice at Google. I don't volunteer to give the interviews, though, because the interview I was trained to give, I would never pass.

And sadly, this is common and recognized at Google, was mentioned in interview training, and is just said to be part of the fact that the bar gets higher, or something, mumble mumble.

I find this disturbing.

Every year I've noticed the questions get harder too. A hard question in 2012 (off the top of my head, 2 pointer solution to linked list cycle detection) might be considered an easy question today.

Thank you for posting this.

My experience has been quite opposite.

I worked for a FANG and conducted over 100 interviews. Most of that time I thought I was asking really hard questions and wasn't sure I would have been able to answer them if I hadn't seen the question before.

I later interviewed at a different FANG, surprised myself and answered harder questions than I typically gave and answered them better than most of the people we hired.

Not sure what to make of that.

Looks like being an interviewer taught you a thing or two about the test content.

That sounds like when you spend a lot of time exposed to and walking through certain types of problems led you to getting good at solving those types of problems.

The same is true for design/ux jobs interviews. Well, at least for me. I got laid off a couple of months ago, and have been interviewing pretty steadily since then. I have yet to receive an offer.

I already know where my strengths and weaknesses are. I'm fine with 1 on 1s and all that, but when it comes to the technical part of the interview process like a take-home design project or a live design challenge, I usually fail.

I know I have the skills. My work has always been well-received, and I've always received good marks on reviews and feedback, but I haven't been able to translate it to the interview process.

I've tried to assess where I go wrong, and I think I panic and put too much effort into trying to figure out what they want to see or come up with something innovative rather than just doing the work and following my normal approach.

The stress of the time crunch (and sometimes not knowing anything about the product or problem) only add to my inability to improve in this space.

When I've sat on the other side of the table, I know that most interview processes aren't that well-structured or set up to actually evaluate the candidate. Even when there's a good process, it's often too easy for one person to influence the results.

So sometimes I tell myself that it was a crappy interview process and it just wasn't set up for someone like me to succeed. But I still end up stewing over it and feeling depressed for a few days.

The depression hits those hardest who put their identity into what they do. Often, I don't realize how dependent I am on positive feedback until it hits. Lately, I've tried to remember in those moments that I am a diverse person and that if I looked in the mirror and saw the worst developer ever I would be OK

If you’ve been looking for a while I think it’s totally normal to get some deep depression regardless of how you identify. Interviewing is stressful and draining and feels incredibly arbitrary. Also, at least in the US, your employment is absolutely tied to your quality of life; unemployment benefits are a joke and you’re on your own after they run out. I’d be depressed as well.

Not saying I disagree with you though, I think identifying heavily with your job makes it even worse.

That's a good point. I probably put way more into my identity as a designer than I should or care to admit. Thanks, gives me something to think about and work on.

Hang in there dude, I think the coronavirus and remote working really messed with the traditional hiring process.

The fact that you are getting steady interviews speaks for itself that you have the design chops and experience. I think people don't know how to interview design candidates well as is, removing the physical cues you get from an in-person interview makes it even harder to assess a designer. Also, it could just be companies are flying by the seat of their pants and a lot have reduced workforce with this second wave of infections.

High probability it's not you, it's them and their broken process during covid-19. Keep your design skills sharp and keep building your portfolio and use this time to work on concepts that you are passionate about that haven't come up in a 9-5, that will help you until the right opportunity comes along. You got this!

Thanks, I appreciate the words of encouragement!

>I panic and put too much effort into trying to figure out what they want to see

My approach for take home challenges is this. Often I complete them and never even get any feedback from the firm beyond a form rejection letter. If that is going to be the case anyway I might as well make something I'm proud to have in my portfolio, even if it doesn't line up with exactly what they are asking for. That transforms the work from something that I'm doing for them to something I'm doing for me. I know that's hard when you are unemployed and have bills to pay, but if at the end of the day you have some work that at least meets with your expectations you can retain that bit of your pride.

This made me think, what if we post take home assignments on github. Along with company name and status - accepted/rejected. This is a testimony of your work as well as of hiring company - if code is good and you are still rejected, that is the red flag to developers.

For a live design challenge I recommend slowing yourself down and asking for more input from your interviewers. If you appear relaxed and asking for input from others then you will seem more open to both collaboration and constructive criticism. My two cents and good luck.

This. Adding up, front-end engineering has an identity crisis now. A friend of mine is a web dev who started back in the days when Javascript was for form validation.

When JS frameworks and SPAs took over the web, he adapted himself. But being a UI engineer he cares about the craft's visual aspects, than the technicalities of JS.

Although he has a proven record of his works, he is now dealing with interview questions like "what are the problems with closures" and "when you do use function.apply()" and "immutability".

Do you have a (substantial) GitHub? Why not mass-apply for positions and refer people to that instead of accepting laborious technical tests?

Additionally, companies doing common pool hiring are worsening the situation even further. One bad interview day or one failure to solve complex riddle means that you're not good for any teams within that company until cool-off time passes which is typically from 6 months to 1 year. The system of cool-off period is equally brutal.

I was on the other side - I flew out to FB during a vacation earlier this year. I got there, could explain the solution but something was off and I kept losing the thread when I went to convert it to actual code.

The interviewers didn’t know I’d gotten a call from my cat sitter that morning and one of my cats had to go in for an emergency check up. (He ended up being fine but didn’t know that at the time).

Later that week I got the rejection from FB and then, 15 minutes later the positive response from Google. Really made me realize how much impact the day has on an interview outcome.

I mean this would apply to going on a first date as well. First impression is what matters and once the impression has been ruined, all bets are off.

It is not specific to the interview process.

Rescheduling a date or an interview at the last minute doesn’t make a great first impression either.

It doesn't give a good first impression, but failing miserably is somehow even worse...

I rescheduled the second date with my now wife at the last minute. Worked out well for us, but I was so tired from work that there’s no way that date would have gone well (busy period of a project at the time).

I would LOVE to get my 2 hours back in a day because of a last minute interview cancelation. The only person who would be truly irritated is the recruiter, who's job depends on meeting candidate quotas. Even then, they want you to be successful.

Trust me anyone reading this, if you are not feeling well, or something is distracting you, please, please, do feel free to reschedule your interview, even last minute.

Once you go through the process, and it's a no-hire, that is recorded in the system and you won't be allowed to try again for some extended period of time. At my company, it's 1 year.

This sounds like sth it could be good if companies informed the applicants about, hmm

I think I too wouldn't have "dared" to postpone, in some/many cases

For a small company it matters, but for a FAANG interview the person rescheduling your interview is completely disconnected from the people making the hiring decision so it would literally not make a difference.

The obvious counter to this is: Which will give the worse impression: Canceling for a good reason or showing up and being distracted and performing poorly? Companies like the usual FAANG are nice enough to merely give you a 1 year embargo. In other companies I've worked at, you interview for a team and if you do poorly, the team will simply never interview you again (although you could interview for a different team at the company).

Also: Do you really want to date someone who doesn't care that your cat may have a life threatening problem?

Finally, even if it's not that serious (cat emergency): Do you really want to work/date someone who is that susceptible to this bias? I know it's the norm to fall for first impressions, but for me, it's also a signal of problems I'll have with them in the future.

BTW, my cat had a serious health problem once. She needed lots of immediate care for a few months (at home, away from work). And the vets were telling us that even if she got through it for now, it will come back soon and we should really consider euthanasia - the condition would eventually kill her - not many cats would last a year with it, and chances are her she would be in pain for much of it.

And this all started a few days before I started my new job.

I went straight to my new manager (before my official start date), and explained to him that the cat was my priority - I could take a leave of absence or whatever was needed, but I needed to figure this out (either euthanasia, or time off for treatment options, etc) and could not be anywhere close to 100% at work.

Fortunately he was sympathetic and said I should not worry and do whatever was necessary.

The job turned out to be crappy, and I stayed there longer than I should have entirely because of this.

A pro tip for anyone reading this who ever ends up in a similar situation: just postpone the interview. I promise you that everyone will understand. Everyone gets that these things are a little goofy[1] and wants to set people up for success and not doom them to failure.

1. Yes, there is a larger and more complicated discussion to be had about making the whole system better. But that isn't gonna happen in the short term. This is advice for dealing with the situation as it exists today.

Yeah I didn’t realize how distracted I was until I was there and just blanked.

I also have real problems with traveling to interviews since my other half has issues being alone at home hence doing the interview while we were visiting her family. Even then me being gone was highly stressful for her.

Google was willing to let me interview with folks local to us which really helped. I’m curious if more companies will conduct remote interviews in a post Covid world too.

If you explain extenuating circumstances to your recruiting contact, you probably can get a chance to reinterview sooner than the usual cool off period (especially if you did well in the earlier rounds).

So far I managed to avoid traveling for interviews by just telling them that I prefer doing it over the internet.

It's worth emphasizing, as a hiring manager: By the time you're in an interview, I want to hire you. I'm sitting there hoping to write "Strong hire" on my feedback form.

I think this is true as a hiring manager, but isn't necessarily true for an interviewer, who may be a senior engineer applying a very strong filter to the "who would I want on my team" test.

Same. If someone makes it that far, I (and several others) have already decided that they stand out. So I'm secretly rooting for them the moment the interview starts.

Uh, unless you saw a better candidate earlier in the day/week.

I'm senior enough that if I rate two people strong hire, I'm capable of getting more budget. This has in fact happened recently -- we couldn't decide between two great candidates, so we offered and hired both.

That depends on the size of the company, no?

Yes, and in some cases, the company needs one person now, and another one a bit later. Then can be nice to have a list of people who did well, to contact again

Sorry to hear! I have a similar (sort of) story.

While waiting at UBER reception area I got a call from a "tax office" telling me about suspected tax fraud transactions (it was a fraudster call but imagine me at that moment) a minute before I was called in for an interview.

Yeah, I was really kicking ass at that interview (NOT!)

I think knowledge workers will have to learn more about human psychology in the future.

I know I go thru waves of high productivity and low productivity. That productivity seems to be closely related to the ability to concentrate in my environment, sufficient sleep, how closely I've followed y daily rituals, fewer stressors in the past few weeks, etc.

I'm pretty angry with the person that you were. I'm frustrated with the people who look up a question, see the exact solution and walk through and then expect the candidate seeing the problem statement for the first time to walk through it in the same way.

There are questions being asked that full academic papers have been written on. What's even worse is typically I get interviewers that are not experienced+prepared enough to give hints or work with you.

I am happy that you've had an epiphany to see where you were wrong. If you ever are in a situation to build a quality team. Interview people for the skills to work with others+ teach others, look to optimize, experience, and that they're willing to learn. You'll have a fairly good group of candidates to pick from as that don't do well in these coding interrogations.

I bomb on technical interviews the same way because my mind just freezes and can't get through even simple problems.

I had the same question you mention. "This is a problem a paper was written on, are you expecting me to derive a whole paper from first principles in half an hour, or are you expecting me to have seen it before? If it's the latter, why not let me Google, as I normally would?"

I feel that saying "I would solve this problem with <algorithm>" should be good enough, and if I'm asked to implement it, I should be able to copy paste the Wikipedia pseudocode and start converting.

The funny thing about this is:

They're not looking for someone that is eager enough to show they're willing to work through it even though they have no idea to proceed.

They are looking for someone that got lucky enough to be ready to answer that particular problem, but won't stand up to them that the ask was a bit too much for a reasonable interview that is intended to see if you're a fit for the company and skilled enough to get the work done.

They aren't looking for someone that will dress them down to tell them that's incredibly insulting to ask something that took an academic a long time to come up with a solution for that. [Although a rough approach and fairly unfriendly. That shows a lack of desperation, confidence on the knowledge, and a good understanding of the difficulty of the task at hand (heck that's a good signal for won't underestimate points during planning)] (There's a singly linked list question that amazon used to ask that qualifies for this)

Exactly agreed, and because my inclination is to say "I've never used this in previous jobs and will never use it here", I don't think it's useful for me to be interviewing for jobs.

A friend of mine, when asked to implement an AVL tree, actually asked an interviewer when the last time he had to implement an AVL tree on the job was. He wasn't hired.

They must be willing to eat their own dog food or else the question won't make any sense for the interview.

The department should run the interviews through their own employees first. If some questions cause an "interview anti-loop" (a set of other employees S who would not hire E) it's time to revise those questions.

Revise and rehearse these practice interviews within the department, and do it blind, until all of S are willing to "hire" each other for their respective roles.

I've never had this prep for interviewing people.

Most of the time you see the resume a few days before. They may say a few things "don't say this don't say that". (hr)

Most companies take the rejection as a point of pride. "We've rejected 100s of candidates."

Asking you to visualise & manipulate a novel data structure you don't have practice with is a feature, not a bug. It's a specialised IQ test.

And what is asking you to reinvent on the spot an algorithm that took scientists years to come up with?

Very stupid, and shows the interviewers don't understand the point of the process either.

I don't like the structure, I think it's very flawed, but asking you to do something you do regularly defeats the purpose.

Don't forget, not only are they looking for someone who already just knows the answer, they frequently aren't hesitant to penalize someone who says "Fair warning: I've already seen this problem."

I don't think it's that complex. I think they're just cargo-culting. Technical interviews were designed to test certain things (brainpower, grasp of fundamental data structures) and not other things (workflow, ability to write out large volumes of trivial code). I think companies just copy that and don't realise what they're really testing for.

> I had the same question you mention. "This is a problem a paper was written on, are you expecting me to derive a whole paper from first principles in half an hour, or are you expecting me to have seen it before? If it's the latter, why not let me Google, as I normally would?"

> I feel that saying "I would solve this problem with <algorithm>" should be good enough, and if I'm asked to implement it, I should be able to copy paste the Wikipedia pseudocode and start converting.

Long ago, milo.com posted a challenge to codeeval.com (both websites now dead) which turned out to be an example of the "assignment problem". That is, given a set X, another set Y, and a payoff function f(x,y), assign each element y in Y to exactly one element x in X such that the total payoff from all assignments is maximized.

I tried to solve it, realized I didn't know how, and was eventually able to look up the solution in a couple of papers. There is an algorithm called "the Hungarian algorithm" which will do it.

So I wrote up the Hungarian algorithm and sent it in. This was good enough to get a phone interview with the company.

I tried to prepare for the screen by going over the Hungarian algorithm to the point that I could myself give the proof that it correctly solved the problem. This was fun. But in the phone interview, it was barely mentioned at all. According to the interviewer, I passed that step by just mentioning the name "Hungarian algorithm".

That's better than asking you to come up with the complete solution, but it's still a bit of a trivia question. On one hand, it's good to be able to recognize algorithms so you can look up their solutions instead of wasting your time on trying to reinvent them, on the other hand, how often are problems this algorithmic?

Maybe I've had a particularly depressing career, but 99% of it consisted of various ways of interacting with a database.

> That's better than asking you to come up with the complete solution, but it's still a bit of a trivia question.

How? You have unlimited time to find the answer; it happens before you make any contact with the company at all.

Oh, I misunderstood the situation, sorry. I thought it was on an interview.

Yeah, in certain circles the hard part is figuring out the solution is the assignment problem. Then you’re supposed to just know that the most commonly used algorithm for it is the Hungarian algorithm, maybe its complexity and that’s it, because you can find code for it in most languages and so you just need to call the function with appropriate inputs.

I had that exact same experience with milo.com. Small world.

Well, I found their challenge because they posted it to HN.

Everyone inside Milo seemed to think that their hiring process was hugely better than what happened elsewhere, including by producing better hires. But the approach -- set a difficult challenge instead of an easy challenge, such that, if someone passes the challenge, you will almost certainly hire them -- still seems to be vanishingly rare.

Oh, was that their logic? I don’t even recall if they actually interviewed me, despite passing their challenge. As I recall, I was rejected for not enough experience.

You're misunderstanding what they're trying to test. They're testing your raw ability to manipulate algorithmic structure in your head. It's important that you have to visualise the data structures and manipulate them without being able to look it up. It's an brainpower test, not a workflow test.

Which is fine if your job is really going to be centred around developing algorithmic software in a unique or high pressure way. The problem is most software engineering day jobs are nothing like this, so the test is not a good filter.

I mean, I can see why they would want to test both. Horsepower is a good indicator.

I always say that you aren't qualified to work on the hardest problems you can solve, so to check if someone can do the job you need to ask them to do something harder than you expect them to encounter. People who work at the edge of their ability makes tons of mistakes and are very slow, and it isn't very fun for them either.

It is the same as if you expect a warehouse worker to regularly lift 20kg then you don't test your hires if they can take 20kg at their best, because then you will get a lot of people who simply can't do the job for 8 hours a day. Instead you'd make sure they can lift 40 kg or so to ensure that lifting 20 kg regularly wont be a problem.

Yeah maybe, but these hardcore algorithmic "leetCode" things are like asking the warehouse worker to design a logistics system instead of seeing if they can carry 20kg boxes. The vast, vast majority of software jobs out there involve more soft skills (requirements, planning, scrum and other processes) and "plumbing" libraries/frameworks/data layers together than implementing algorithms from scratch. I'm not saying you'll never have to do it, but it's going to typically be a small proportion of your day job, so why test this stuff so much at interview stage?

The point is that designing an iteration that does something should be a trivial operation requiring no thought at all. If you can leetcode problems then we can be sure that you can do loops without effort, otherwise it might be that you struggle with them and as a programmer it will really hurt your productivity. Testing if you can do loops doesn't test if you can do loops easily, so we test something requiring a bit more thought.

Edit: And I think that you'll find that most leetcode problems are pretty easy if you are an expert at doing loops and recursions.

> I'm not saying you'll never have to do it, but it's going to typically be a small proportion of your day job, so why test this stuff so much at interview stage?

It isn't a small portion of your time spent at Google at least. Almost no engineers at Google ever talks to non technical people, product managers or engineering managers does that. Most engineers works to reduce cpu usage of backend servers or build scalable features for backend servers or similar stuff. Even high level decisions like deciding what service to use requires more engineering hours in time spent refactoring the thing than engineering hours spent deciding what to use, so there is much greater need for people who can code than people who can make decisions.

And if you argue that Google is bad at making new good products, that isn't the fault of their software engineers since they don't decide what to build. Instead it is the fault of VP's, and those don't do white boarding interviews.

So much agreement here. Who hasn't been rejected by some 25 year old who's been with the company all of 6 months and who asks silly questions straight from LeetCode? I have literally been asked questions that were PhD theses 30 years ago.

How is any of that supposed to be part of a good interview process? My only solace here is that those things are red flags on the company side, so I might have dodged some bullets.

Wouldn't it be a more balanced approach if the technical problem were presented on the spot to __all__ interview participants, not just the candidate?

It's a technical job, it must be a collaborative effort in finding solutions! Aren't we looking for a team member, not a lone-ninja.

Sure, the candidate has to be more vested in advancing the process, but the fact that there's no known-ahead answer would potentially surface not only technical skills, but also personal ones, and ... at both ends of the table.

Not to mention, the "house-experts" could find such sessions stimulating, to say the least... Imagine, you get back to your desk after interviewing a candidate, and your team mates ask you "so what was the tech problem this time?", "could you crack it?", "did he/she crack it?".

Very intriguing idea. I’m thinking about how logistically this could actually work. It would build a sense of camaraderie as well.

I like it. However, I have had an interviewer at a "top chicago tech place" (roll your eyes on that one) He basically did an inperson imitation of leetcode. Say problem, look at you. Don't give help when asked, stay emotionless.

That's why I think take-home exercises are a much better way to evaluate candidates. Give them the exercise and enough time and you will figure out:

1) How good they are 2) How much they actually care about the job offer -- the more detail and the more passion they put in the exercise, the more into the job they are.

Totally agreed. I've been in the position to evaluate several take-home exercises during the years and they always produced good results

Doesn't that tell you that the method of hiring is flawed in the first place?

If you care to weed out bad candiates, what help are 100% perfectly fine take-home exercises? The only thing that tells you is, that all of your candidates are viable in the first place. Just go into an interview with THAT assumption and ask the right questions.

Why do so many companies assume that people who try to get hired for a certain job don't meet the basic requirements? That gotta be the rule not the exception? That's just a bad faith assumption.

Just make sure your hire fits into your culture and give them sufficient probation time to grow into your codebase.

> > produced good results

> what help are 100% perfectly fine take-home exercises? The only thing that tells you is, that all of your candidates are viable in the first place.

I think you might have interpreted that, in a different way than what was meant.

Could you provide a bit more info? How long were the exercises? Were the applicants evaluated more based on their results or possible writeups? How much did their CV weigh compared to the take-home exercises?

Give 20 candidates take home exercises, they're all passionate and put a lot of work into it, then you'll reject 19 of them.

The input required for this type of interview is heavily skewed in favor of the company. No thanks.

Why not drastically different approach - ie. instead of active technical interview process on company's side, just put up few dozens of open source projects/libraries that are being used inside company and leave it as technical interview sandbox.

Keep brownie points on pull requests to unlock "you gained <<passed technical interview>> badge" a'la stack overflow or do one off evaluation based on contribution when candidate applies. Stack overflow contributions can be equally taken under account to gain points and arguably they have badge system that does the job for you already.

Github could introduce something similar and wipe out "anxiety interview" completely, if they wanted to.

There is really no reason to do technical interview when you can inspect open source work. For people who don't have contributions yet, open source libraries/projects by the company should be made available.

Seems like win-win for everybody.

Unless you are looking for a "senior developer with x years of experience". I have a live, hobbies and family. I don't care to spend most of my free time fixing code in some obscure library I'm not ever going to use only to stand a chance to get a different job in the future.

How in the world did current developers EVER get ANY job? Who gives developers the chance to grow these days?

It's a "foot in" problem, I feel. Shit companies have shit interviews and give out shit workloads to MAYBE get a foot in the door.

You can follow along or be proactive, for instance start a new github account just for interviews, do code wars challenges, write a blog and just link your writeups, github and blog to applications. Along with your CV with major projects etc. of course.

Most people I know who work at big firms got calls from friends or friends of friends to interviews, so spending time socializing the local hacker circles is probably going to be worthwhile as well.

If you have life, hobbies and family, you should still be able to schedule 1-4 hours a week to job hunting, much more if you don't have a job of course.

It's not pretty or much fun, but it's certainly doable.

Not the worst approach, and as long as it's in the spirit of cooperation might even be good.

UX people could tune or at least comment on the interface, security people could fuzz or pentest the library... however, for a lot of actual problems in actual open source libraries it requires a crapton of internalization -- again, for free. Granted, it's for open source but it could be argued it's even WORSE approach for the company, since they're getting free benefit from expertise and work.

Also, I'm a private person, I have splintered online identities, I'm not going to give my 'hobbyist' github account in a professional context, and I'm also certainly not going to make a github account just for interviews and applications.

I'm lucky enough to have connections that will probably keep me employed as long as I like, and the tales of interviews and free work that's required to put up just to get a chance seems insane.

4 hours... okay, I can sort of see that, I might set myself a limit of 1 hour and tell them this was my approach, this was how far I got. I've heard of 30-60 hour workloads given to applicants to 'top five' companies and it's just ridiculous. As are the stories of interviewers who have no technical skills themselves and are just looking for a canned response (Google, looking at you).

I was going to say that the best approach I've seen are companies that put up interesting puzzles and security challenges on their website. The technical skills required aren't _that_ high, but certainly require an amount of ingenuity, persistence and just plain being interested in tinkering that they feel suits their company profile. The applicants can solve or try to solve these challenges, do writeups and probably get interviews just based on those.

Granted, I can also understand someone looking at 50 companies all giving them 4-10 hour workloads to MAYBE get an interview might feel frustrated and overloaded. It's a classic egg-and-chicken problem. If I were to start looking for a job from scratch I'd probably solve a bunch of code wars challenges and security CTFs, make a blog and just link my writeups and blog to applications.

Aren't you worried about filtering out good candidates who have enough options that they won't spend time on your test?

It's not just nerves either. All interviews, but especially FAANG ones, are random. The interview might be racist, sexist, etc. They might have gotten divorced that day. The question they ask you is something you haven't heard of but they spent their PhD on. You can even do everything correctly, but the interviewer doesn't like the way you did it.

The leetcode stuff is just masochism. You will spend X months beating DS&A into your brain until someone can pullout a random question and you immediately know what questions to ask, what steps to take, and how to write it out on a whiteboard.

I agree with you for the most part, but Google goes out of its way to force the interviewer to just hand in an emotionless, gender-neutral description of the candidate's code and coding performance to a hiring committee that makes the decision

This kinda bit me in the rear when I applied to Google a couple of times. As a nerdy American male, aged 22 to 40, with above-average people skills (for nerds) I got along great with everyone I talked to. But the hiring committee would get a sloppy page of half-functional garbage code, along with a note like "Seems like a great candidate". I found another FAANG company I fit into a lot better anyway, so it all worked out great

It's difficult to remove that randomness from the process. As I see it the only reliable approach for both sides, given that structure, is volume - you just do/conduct a lot of interviews and evaluate/get evaluated on the top percentile.

I've heard HR optimizes processes that limit Type I errors (false positive) at the expense of Type II errors (false negative).

I've often wondered if this is because, from the HR perspective, false positives are much more costly. I.e., Those who would be helped by limiting the false negatives are project managers who are too downstream in the process for HR to care.

I think the problem is HR itself. Few companies should be large enough to warrant full time HR, but even small companies need many.. HR are specialist in expanding the disconnect between people the company thinks it needs and people who are willing to work for it.

You say that, but look through this thread. HR has a solid understanding of what a given company's large-scale objectives are when hiring. As evidenced by most replies in this thread, technicians on the ground do not.

The evidence is that it is a hard problem that is not solved. No amount of people who can't do the task demonstrate the existence of experts who can. Google and Microsoft have shown us examples of failing at simple HR tasks for years or decades with budgets for HR that dwarf a small companies entire payroll.

What virtually every attempt shows is that it is better to hire people who are roughly qualified and see what happens, managing the results instead of filtering. Having an HR strategy for hiring leads to creating biases (i.e. requirements based on your current work force) that gets you a group of overly similar and therefore collectively incompetent work force.

Yes this. Hiring the wrong person can be ridiculously expensive, especially if they linger on in a role for years corroding the culture.

obligatory request for citation

Not trying to be snarky but isn’t it intuitively more expensive assuming you’ll try to fire and then rehire correctly? It’s like asking “why does it cost more to go through a process twice rather than once?”

This isn't StackExchange.

Do you really need a citation for this obvious fact?

You only need to fill a position with a person that is good enough.

If you reject someone who would have been a good fit but still hire someone good eventually, that’s ok.

A bad hire is not fun, not for HR, not for the hiring manager, not for the colleagues and ultimately not for the person who got hired. Especially if they had to move for the job, possibly with family.

A bad hire has massive negative consequences for many people. Of course you want to avoid that, if that comes at the cost of not hiring someone who would have been great occasionally, that’s unfortunate but acceptable.

I don’t disagree but “good enough” comes with its own risks. That may be fine for an organization that has a lot of inertia and is biased towards the “maintaining” side of the house. However, I think it would make it make things more difficult if the intent was for bold, transformative change.

The bar for "good enough" can of course be quite high.

If you're hiring people at the very top or with rare skillsets, like building browser engines, building databases or people whose skill is at author/contributor level to relevant technology (for the job in question), you're probably not going to go with the usual process anyway as there is less uncertainty.

Fun fact: Google & other tech giants have poached a lot of game engine designers to optimise datacenters at truly obnoxious salaries, because their fundamental skill is making complex systems of interacting entities run as efficiently as possible. It's causing problems in the games industry because they can't compete.

Understanding this is the most important part of interview process, in my opinion. Large companies want to minimise bad hires, small companies are more open to odd CVs and novel approaches. Market yourself appropriately.

Where engineering fields have a disproporationate number of individuals with ASD (née aspergers), and where anxiety plays a large part of the syndromes, it's no surprise that many, many good engineers are terrible at interviewing.

I am focusing on the algorithmic part, which is the only thing I can influence in my company's interviews. I wish we would evaluate candidates with some pair programming, but that's never going to happen.

> incredibly high stakes algorithm riddles

Yeah there you go. This is your problem (Google/Facebook I assume?). It's more than questionable to expect candidates to solves complex algorithmic problems on a whiteboard in 45 minutes. You should ask moderate algorithmic questions that offer many solution approaches and accept solutions that are "good enough". It is not about having a candidate gaining some magic insight that some MIT student had 30 years ago during his master thesis (and reproducing it in 20 minutes lol). It's about giving a reasonably complex problem, that is not too easy but also not too difficult and allows you to focus evaluation on:

* Does the candidate write proper code that isn't far from compiling? (If they can't, they don't have the experience. It's like not being able to write without a spell/grammar checker. Small mistakes are fine. Big mistakes indicate lack of understanding.)

* Does the candidate have a structured approach to problem solving (all the time I see people starting to write code immediately and getting completely lost in what the problem even was. This is a red flag to me and baffles me each time again.).

* Does the candidate debug his code and walk me through. Does he find obvious bugs while doing so and can he convince me that his code works? (If not, and that happens often, its another red flag)

* Can he rank the speed of his solution with other theoretical solutions? Let's say they found an N^2 algorithm. I usually ask if there is anything faster (even if there isn't). This shows if they have some decent fundamentals in CS and are able to think about the boundaries of an optimal solution and how far they are from it. This is something, people without a CS major usually can never do and unfortunately also not too many with a CS major. It's kinda relevant though, if you optimize for performance and have no clue what the theoretical limitations are, then you are grasping for straws.

There is one big secret for getting A LOT out of easy questions:

I start to modify the problem statement and see if they understand how this changes their solution and their algorithm. People who don't have a good grasp of CS will fail miserably at this task, because this isn't something you can memorize.

pivotal does pair programming interview. The morning with one person in a theoretical problem and afternoon with another in a real problem.

I liked the experience and the process, in the morning it was a blast. We really clicked.

In the afternoon I had to use the same setup as the guy I was collaborating with -- Vim with his bindings and Golang. I had never used Golang before and even tho I was proficient with vim his binding were quite opinionated.

I felt the guy from the afternoon had 0 interest in being programming with me. He wasn't collaborating and was looking at his cellphone the whole time.

The problem was to write a functional test for a CLI that would spawn a web service in Golang, parse some json and run some assertions.

Golang had some quirks with JSON and new line encoding that I spent my whole afternoon with. It is something he could have unblocked me as it a very specific problem, instead he kept telling me to read the official golang docs, not even to google the error.

As someone being interviewed I followed what he asked me to do and worked in his environment but it wasn't a representative of how I would have worked.

That said, I feel it could work! I just should be able to judge the person who pair programmed with me instead of just being judged :) the person in the morning would be a 10/10 and in the afternoon a 4/10.

You've just slightly improved upon a very broken way of interviewing engineering candidates. I've been paid by clients happy with the results to do everything from cryptography to writing a compiler / interpreter. Very computer sciencey stuff at times. The devs I work with consistently rate me highly and I often get put in mentoring positions. I couldn't tell you what N^2 is. I learned it in uni. In decades of programming it has come up in many interviews and never once on the job. Maybe in your industry it's important. But if not, please consider re-imagining your interview process. Having hired many developers over my career including at one time for my own business I can assure you there are far better ways at getting high quality signal out of candidates than trying to improve upon what you experienced as a candidate.

"N^2 is bad, except when it isn't" is my take on it.

In uni I maybe had one week of doing O-evaluations, it never came up and I never got interested. I've seen 20,000-word debates on Reddit on whether something is n, n2 or logn... and to my knowledge nobody ever learned anything from those.

In the workplace I've several times ran into the problem of "This is taking way too long," developing tools and methods to measure and drill down, then figure out if it can be improved upon or should be left as-is for now.

Honestly discussions and articles like this leave me absolutely terrified of interviews. My first and only technical interview was my future boss leaving me alone for 30 minutes in a room with a laptop "Code something you like yourself."

That man was brilliant.

The problem with Big O complexity is that it just isn't the reason most things are slow. Usually, your input is either so large that you physically can't run a polynomial solution, or your input is so small that it doesn't matter. Speed increases usually come from caching, avoiding indirection, removing slow API calls & properly making use of your hardware.

> In the workplace I've several times ran into the problem of "This is taking way too long," developing tools and methods to measure and drill down, then figure out if it can be improved upon or should be left as-is for now.

To play devil's advocate, someone with the word Senior in their title would probably have command of recognizing Big O issues, and skip the "this is taking too long" step, and code it efficiently the first time. This is industry dependent, as well, obviously. I've been doing low-latency C++ for 15+ years, and it's not really something you can compromise on.

The mistake, I think, is skipping otherwise good candidates because they don't immediately see these issues. We should put more emphasis on identifying these weaknesses and finding the right mentors to teach them up. We should be hiring more junior and mid level engineers, with the assumption that they will learn and be up to speed in a few years. This has been my approach to interviewing, but I'm often vetoed by other team members.

It's also sort of ironic how studies repeatedly show high preforming teams, are cognitively diverse teams. Even though research shows this, the hiring strategy of most technology companies, is hiring new employees with the same skills, and traits, as current employees.

https://hbr.org/2017/03/teams-solve-problems-faster-when-the...

This is a critical point that a lot of hiring misses. We talk about building great teams, but the hiring requirements are static and never seem to consider who's currently on the team and what mindsets or backgrounds are missing.

Senior/Principal Engineers at FAANGs may not even code all that much in their day to day. The difficult decision that gets faced when interviewing those folks is whether they could walk into a fresh environment, and stack, and immediately earn the trust of their team.

In practice whiteboard interviews assess this pretty well. A new senior engineer must immediately be shipping optimal, tested, and well designed code. Otherwise you'll have Junior Engineers coaching Senior Engineers and the Junior Engineers will just get frustrated and leave. Worst case, you'll have an "architect" who never learns how to actually deliver independently in the environment.

Any significant system will take months to learn. Your statement is probably only true for some trivial system.

A senior engineer comes with years of experience designing and coding systems that took years to develop and run at scale. Given time to learn they should be just as good at solving day to day bugs but if you assign them that you are wasting your time and money on experience. If you don't value experience you are wasting your time and money on experience.

It's pretty normal to have more junior engineering staff help new senior staff learn the ropes. It's also common for those less experienced staff to learn from the more experienced people even in that situation. I'm not saying senior hires should be respected from the first second because they are over 30 or have a long resume.

Learning the ropes of a new company is fair and being slower at first is to be expected, but explaining that code reviews need to be tested to a senior hire isn't as fair. Similarly a new senior hire should be able to hop into a code review and catch problems/suggest improvements early in the process.

These two behaviors aren't as dependent on knowledge of the system as they are on core coding competencies. The whiteboard interview reasonably works to assess these, but it requires a lot of prep and is prone to false negatives.

Have done over 1,000 interviews for a FAANG competitor that IPO'ed in the past 10 years. Completion time (in minutes) of a coding puzzle is essentially the only objective measurement taken during these coding puzzles, and is sadly the primary datapoint fed into the hiring manager's decision. The hiring managers usually adjust the cutoff based upon whatever quota they want and/or if they're expecting people to leave. I've worked directly with a VP of Engineering and that's really all he wanted to hear.

The whole process is missing a feedback loop to the evaluators, resulting in the OP's situation. In my own situation, it took several years after that mess of being forced to shotgun candidates to pick apart what in the experience was actually predictive of success and what wasn't. To this day, the most successful hires from my perspective were those who just really wanted the job and wanted to contribute to the product; definitely no strong correlation with "leetcode" ability.

It's not just that the coding quizzes are poorly administered, but that the hiring mangers and leadership industry-wide have been obstructive to getting evaluators feedback (for starters, what did a candidate's other offers look like? where were they 3 months after the interview?) and especially tight-lipped on sharing interview questions. That last bit is particularly obnoxious because people monetize their insider knowledge either through things like Rooftop Slushie or they take a more Gayle Laakmann angle. (Just to be clear: while many questions are "protected under NDA," many questions are also stolen from other companies-- interviewers asking them are already violating their own NDAs when asking them, or potentially when feeding the question bank of their new employer).

What you can do is go to your manager in your 1:1 and ask for interview feedback. Ask what happened to candidates 3-6 months after the interview, even if they were hired (perhaps not just to your team). Start to get feedback, and then work from there. You'll probably do better interviews, and at least curtail some of the information arbitrage culture in the C-suite.

Occam's razor is: algorithm tests are specialised IQ tests, and that version of IQ is actually a very reliable indicator of performance (compared to the other signals available).

I'm not exactly sure I buy it, but it seems like the simplest explanation.

The best advice I ever got about interviews is to figure out ways to prove yourself wrong about what you think about the candidate. Although it's good advice for a candidate who seems too perfect, this advice really shines when someone is bombing.

That advice seems like unnecessary mental gymnastics.

Just take an unseen technical problem, solve it for the very first time in front of a colleague, then double your time as the expected time for candidates in an interview.

When you're under time-pressure, you actually spend more time thinking about how much time you have left, rather than thinking about how to solve the problem.

It's not mental gymnastics. The point is to challenge yourself as an interviewer to reveal a diamond in the rough. Some people suck at interviews and are great at coding. I've hired a lot of them and we've made a lot of money together. I've never had to fire anyone either so I'm not getting false positives either.

> Just take an unseen technical problem, solve it for the very first time in front of a colleague, then double your time as the expected time for candidates in an interview.

That gives very poor signal in my experience due to the time pressure. Your process is now selecting for people who are good at taking tests instead of selecting for good engineers.

I call these high stakes interviews "the perfect dismount".

You can feel the pressure mount - any algorithm that doesn't run on the first try or second (with a couple of quick fixes) and you're being mentally discounted.

The same for silence - if you're thinking through something, take abnormally too long, you're discounted.

The Perfect Dismount. A 10.0 is what's needed to land a FAANG job.

I just pulled out my laptop and if they aren't cool with the solution I google'd in 30 seconds, they can test my critical thinking in another manner. That was my critical thinking and resolve under pressure.

These are nothing but shit test.

What are shit tests? When somebody fucks with your head to see how you will react, what you are experiencing is typically a (series of) shit test(s). Everyone has been shit tested, gets shit tested and will continue to be shit tested; We use shit tests to make value judgements about people, likewise they can be used to determine how people cope under pressure. The underlying mechanism of shit tests is to test your mettle.

What some of us do now is at home exercises with no deadline and setup the interview to do a code review of it after it's given back.

It gives so many advantages: - people will have some fun trying to solve a real issue / build a little software - they will show how good they are at stack overflow, general methodology, cleanliness, test coverage, deployment and even git (how many we see who can't even gitignore their project files, make a build script that works, or test for real) - horrible candidates who can't even cheat will not be able to lie their way around a fake resume, cheaters will bomb the code review quite fast - candidate will also discover their future team mates general work attitude during the review and if the reviewer is good, will lock the candidate faster than "how many billard balls can you put in a Boeing 747" type of interview. - candidate will even LEARN during the code review => you get away with something out of the interview even if you fail

Since I've been interviewed that way, doing my little project on my wife's pregnancy bed smiling like an idiot at solving the challenge, then fell in love with my manager showing me new tricks during the review, I'll never do any other kind of interviews myself.

I recently did one of these where I was paid at half the hourly rate of the job offer. There was an initial unpaid assignment that was difficult but totally related to most engineers' day to day work. That took less than 30 minutes. Anyone who passed that got the paid assignment which took a few hours.

If I had options I wouldn't even do a 30-minute assignment. I'd just go with someone else. I'm reading about employers giving assignments a lot in this thread, but what do you do about the fact that the people you really want just don't have a reason to waste that time?

There was no interview. Not even a phone call. After I submitted the 3 hour paid assignment they hired me by giving me access to Slack and we just started working together. One of the smoothest and easiest "interviews" I've ever done and I got paid for everything except for a 30 minute challenge that helped solidify my understanding of git.

> people you really want just don't have a reason to waste that time

I'm "really wanted". In the past week I have had to say no to two clients calling me for follow up work. Where do you think I wasted my time?

This seems like a very nice process.

It was great and I enjoyed working with them. That's why even though interviewing is broken at most companies I enter the process assuming the best and with an open mind. I'm disappointed 90% of the time, but you don't want to throw out a gem just because you usually dig up worthless stones.

Couple that with 45 five times over! I was flown down for a Facebook interview from the east coast. It was my first tech interview and it was with facebook. I was so exhausted by it, and then the next day I had an interview at Uber. I started that one already pretty tired.

I much preferred our style of (technical) interviews (I took on a few dozen over the years). First there's a review of the CV, Linkedin, and github, if that looks good, a first interview - just get to know the person, very casual. If that clicks, second interview is the technical interview - we did that by giving the candidate homework, 4-8 hours worth of tasks that are close to their day-to-day work, usually something with a REST API, some basic math/arithmetic, and a front-end / forms, and plenty of room for the candidate to play with.

It was received pretty positively; especially early on (2012-2015) we had a lot of candidates indicate they had never done anything with JSON or REST before. (later on they mentioned having done nothing with XML before, lol).

And we got to see some interesting solutions and creativity; one guy we hired did it all in J2EE, while we were a Spring fanclub. But the code was sound, he showed that he understood how it worked, etc.

The conversation during the technical interviews were entertaining as well.

> basic math/arithmetic

Can I ask for some example(s) of what type of questions you asked?

I learned this lesson a few years back when I interviewed for the CTO role of a educational software company - the main part of the interview was preparing a presentation on how to launch a new product which I had to prepare and present to the management team.

That went very well and I was feeling pretty good.

Then the CEO had a quick chat and happened to ask me a very simple technical question and my brain would not work - I completely failed to do it even though it was trivial.

I think that's how I discovered that mental context switching is a real thing - I had spent a few hours preparing and giving a presentation and then when asked a simple technical question those parts of my brain were completely offline. I think the shock of not being able to do it had a big effect on me and I went to bits (possibly the first time I've ever done that).

> I think that's how I discovered that mental context switching is a real thing

You had to go to that level to understand that? I am glad you did realize it, but I am surprised it was that late in your career.

I'd probably heard about it but I'd never had it really hit me like that before.

How did the CEO react / say? If I may ask

I'm so glad this realization is the top comment on HN.

I think Fizzbuzz can be actually a reverse test: if an experienced programmer can fail fizzbuzz in an interview, then maybe it's the interviewer who screwed up.

For three years or so I taught week long corporate training classes at Apple, Microsoft, Facebook, eBay, PayPal, Cisco and Intel. Every week, week after week, real rocket-science level Android and iOS deep-dive in to the guts of the operating system classes dealing with firmware and device drivers and file systems. In the classes we didn't build apps, we built operating system components. Small to large classes, thousands of software engineers from senior and up, over the course of those three years.

I would stand up in front of 80 or more engineers, go in to esoteric aresa of the operating and talk eloquently and at length about the structures and code found there, and live code on a projector, answering questions about the code and debugging the code of everyone in my class as they followed along. Day-after-day. 40+ hours a week. It was intense. I became an independent advisor to teams and a consultant for some of the groups on how to bring up Android on new devices, create device drivers, and so forth.

Some of those classes, I wondered how some of the people I was teaching was actually able to hold down their job.

Months later, after I stopped teaching, I interviewed at Intel and PayPal and Cisco. I bombed every interview. I was, by all accounts, unable to program my way out of a wet paper bag.

Everyone has off days.

I've done my first programmer hiring for my startup several years back. Thankfully I've came across Guerilla Guide to Interviewing[1] article by Joel Spolsky, and that helped greatly.

I've interviewed 6 candidates within a week, and hired one. Never regretted of the outcome.

I've set up a test code so: an input variable, comparison variable, and an empty function. Function takes input variable as an input, and output of which is compared with comparison variable.

Interviewee would be asked to fill in the body of the function, and play with it until function gives the expected output. I've set up a computer with two screens, put IDE on one, and Google on the other. Two chairs, and some coffee.

First I've chatted with the interviewee for about 10 minutes, tryin to make them comfy as possible. Then before the test, I've stressed very much that I am also a programmer, and I know how awkward it is to be coding while someone watching, and that'd alone would make me do silly mistakes, so I was expecting same from them and that was completely OK.

Afterwards, I've encouraged them to use Google in plenty, and feel free to stay silent or explain as they go.

So I got to assess their fluid intelligence, their ability to break the task down and progress efficiently, their English proficiency[2] (if they use Google in English), their usage of keywords, their choice among in stack overflow responses, etc.

At the end I've rejected a guy with 8 years of experience on his CV, and hired a junior. Looking back, that turned out to be an amazing decision, best I could have made, for work went good with the junior and I've had the (mis)fortune of working side by side with the 8yr guy several years later.

PS 1: Task was to write a recursive function to travelse a multidimensional array, and find out whether the first letter of every "value" was a capital letter. PS 2: The work was to maintain and develop mid-scale SaaS project along with me.

1: https://www.joelonsoftware.com/2006/10/25/the-guerrilla-guid... 2: Country's mother language was not English, and English proficiency was not good on average.

People bomb, including good people. That is sadly part of the system.

Once I had this candidate. Damn, I know my questions are deceptively simple on purpose, but he literally couldn't do anything. Not even a trivial brute force. Not even any related simple knowledge questions. Couldn't tell an average from a median. The only good thing I could write in the feedback was "seems to know some basic syntax".

It was quite a learning moment to read that everyone else was praising how this was the most brilliant candidate they've seen in years. Well, mine was the interview just before lunch, in a schedule that was atypically later than usual. I guess starting an interview after most people start their lunch break was asking for it. He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw.

Interestingly, there's a famous study showing those doing the judging "can't think when hungry" either. (Or, at least can't think unbiasedly).[1]

[1] https://www.pnas.org/content/108/17/6889

http://daniellakens.blogspot.com/2017/07/impossibly-hungry-j...

That study turned out to be quite flawed, because the cases were ordered [1].

In particular, cases were grouped by prison and within each prison, prisoners without an attorney went last. As the well-known saying goes, "the man who represents himself has a fool for a client": the pro se prisoners fare far worse than those with a lawyer. Judges tried to complete an entire prison before taking a meal break, and therefore ended one session with the statistically weakest cases and began the next with stronger set, thereby guaranteeing the result.

If you look at the original data, some other oddities pop out. A physiological process, like becoming hungry, should depend more on the wallclock time than the number of cases heard. However "However, note that in an analysis that included both the cumulative minutes variable and the ordinal position counter, only the latter was significant."

[1] https://www.pnas.org/content/108/42/E833?ijkey=21207497c684f...

Ah, thank you! The more I look into a lot of these (in)famous behavioral psychology studies, many don’t hold up or aren’t replicable. As a layman who is simply interested in the topic, I appreciate the correction and insight!

Well, then. There's the solution. From now on, I'm bringing my attorney to all my on-site interviews. :P

> Couldn't tell an average from a median

Without meaning to attack you personally (especially in the context of the rest of your comment), a comment like this annoys me a bit.

I presume by "average" you mean arithmetic mean, but the median is also an average, and depending on the context the median might be a far more useful statistic than the mean. Confusing the mean and the median is one thing, and perhaps you actually used this terminology in the interview; "confusing" "the average" and the median isn't really worthy of comment, and sounds more like a breakdown in communication between interviewer and candidate rather than a lack of technical knowledge. It just seems vaguely hypocritical to me to be expecting a certain level of ability from the candidate and then using imprecise/informal terminology.

I know nothing about GP, but note that it is possible that English is not his first language, and the subtleties of the story and vocabulary might be lost in translation.

In my native language (French), as you describe it, there is no word for "average" (the technical and the common terms "moyenne" are exactly the same). Until your comment, in English, I assumed you could use "average" and "mean" interchangeably.

My native language is English and I'm fluent and well educated.

Although I do know that "average" covers mean, median and mode, when I was young I was not taught that way. First I was just taught "average" with a formula, and many years later "median" and "mode". I don't think I noticed the term "mean" until I learned about median and mode.

You can see this reflected in scientific calculators used at school, where the button for calculating mean is labelled "Avg".

And in Microsoft Excel or LibreOffice Calc, the function to calculate mean is called "AVERAGE".

So nobody should be hard on themselves for not knowing the difference. You probably weren't taught the difference, and common tools work against it.

It looks to me that the common word for statistical mean is "average" in English, but it's not the correct technical language.

> So far as I can tell, the common word for statistical mean is "average" in English

You're quite probably right about common parlance..

> but it's not the correct technical language.

..but as you point out it's not really appropriate in a technical context, indeed I don't think I've come across the term "average" being used in this way in technical literature; I've normally either seen expectation (for probability theoretic cases) or mean (for statistical cases).

> So nobody should be hard on themselves for not knowing the difference.

Absolutely, which is why I think being hard on someone for not knowing the difference between "the average" and the median is similarly an issue.

You realize that an interview is not the same as writing a technical article, right? It should be a conversation between two potential coworkers, and, in that context, I think saying "average" instead of "mean" is totally fine.

Incidentally, "mean" doesn't quite cut it, if you want to be a stickler about it. There are several types of means: arithmetic, geometric, harmonic, etc. By that standard, you just failed the interview.

> "mean" doesn't quite cut it, if you want to be a stickler about it.

I'm going to assume, then, that you didn't read my original comment which started off this line of discussion? By that standard, you just failed the interview. ;)

Sorry, I dropped this! :P

Touche.

French and English are strictly equivalent.

moyenne is mean or average. The formal name is "arithmetic mean" and "moyenne arithmétique".

median is médiane. people frequently use any word but median in both languages when they want to talk about the median.

https://en.wikipedia.org/wiki/Arithmetic_mean

https://fr.wikipedia.org/wiki/Moyenne_arithm%C3%A9tique

Almost correct.

The mean, median, and mode are three types of averages. None is 'the average'; though if you asked someone random on the street they'd probably give you the mean average.

This really depends on context. When evaluating a likelihood (prediction), "average" is generally colloquial for "expected median."

On an average day?

Mode: On a typical day <<-- most common meaning for average in day to day conversation

Mean: On a day that is in the calculated middle (for some variance) <<-- most common meaning when we think the question is about math

Median: On a day that is the middle for all the possible days <<-- nobody thinks of this, but they think this is what the mean is.

In most practical situations, your initial impression is correct. As a native English speaker, I've never heard seen or heard someone use "average" to refer to a statistical summary other than the mean. If it doesn't refer to the mean, it doesn't mean anything formal/technical at all. I read GP as "median is sort of an average in the broad sense", but this is honestly kind of a stretch.

Math definitions only..

I see it quite commonly, from technical discussions to newspaper article. When people say "the average income in the US" they are almost never talking about the mean.

I think they are. I doubt any newspaper articles use it to refer to the median, at least I've never seen them use it like that. Do you have any examples?

Almost every discussion about average salary or iq uses median. Even finding results for mean salary is difficult. The first page of results for mean us salary gives a list of results for average, many of which specify they are actually the median.

https://www.google.com/search?q=mean+us+salary

Well, https://en.m.wikipedia.org/wiki/List_of_countries_by_average... as a counterpoint. Almost all the Wikipedia pages about average wage use mean.

...I always learned in grade school math class that "average" = "mean", very specifically. I'm pretty sure there would have been test questions that required this knowledge.

It's very possible my schooling was wrong, but presumably I'm not the only one.

I have a graduate education in probability and statistics; in the technical literature, an "average" is an estimator which, subject to some bias, predicts the value a sequence tends to. My professor was fond of saying, "the function f(x) = 15 is an average, but since it's a constant function it will almost always be a terrible one" to drill this into our minds.

The arithmetic mean is colloquially called the average and differentiated from the median and mode, but technically speaking these are all averages. They are all estimators of central tendency. It would be not be out of the ordinary if a statistician asked, "which average do you mean" for clarity. Which definition of average is best depends on what you're trying to measure.

I dislike this interview question because it's an area where it's 1) more trivia than practical knowledge, and 2) easy to not recognize someone who knows more about it than you do. It's like if I tried to test your knowledge of the difference between "affect" and "effect", and then you correctly used "effect" as a verb and I thought you were wrong.

If I repeated everything I just said about estimators in an interview, the interviewer might think I'm too ivory tower to realize that "of course he's just talking about the mean!" But then I could also ask why I'm being asked a question like this in a software engineering interview.

It's a language game where no one wins (somewhat in the sense of Wittgenstein).

> "the function f(x) = 15 is an average, but since it's a constant function it will almost always be a terrible one"

It has minimum variance, how can it be a terrible estimator?.. ;)

Hah, that's a good rebuttal. Well it would be an excellent estimator if it was unbiased (or at least had very little bias). But if f is an estimator for the true average of a function F where E[F] = 100 (for example), it's very clearly not unbiased.

It is a bit tongue in cheek.

> It is a bit tongue in cheek.

As was my reply ;)

It is a very precise function, I'll give it that.

Thank you for articulating this. This issue has many manifestations in the interview process. It can be so frustrating.

All you're pointing out here is that a candidate should be able to differentiate between average, mean, median, and mode, and should be able to swiftly correct "what's the difference between average and median" to "do you mean mean and median? because a median is a type of average".

Given that, showing more charity to the comment you're responding to would have been the correct course of action.

Yeah but will embarrassing the interviewer work well as a strategy to get hired?

Depends on the interviewer.

Do you want to work at a place where you cannot event define the words used before solving a problem?

> People bomb, including good people. That is sadly part of the system.

It seems like there's probably some value in a "MoneyBall" company that identifies a way to interview without most of the anxiety. Perhaps hiring a few people with remarkably high EQ who customize the interview setting who review the inteviewees' work and watch them program over a few days in their own computer and IDE.

I know I've interviewed people that have frozen up, but for whatever reason I didn't pause the interview and address the anxiety, but rather I tried to adjust the questions (which generally didn't work).

These people are usually called "references" :)

As for interviewing for a few days... Yeah. No. Not in a country with an abysmally bad vacation policy like the US.

I do think that the people giving the questions and interacting with the applicant should be different from the ones judging their results. This also gives you the opportunity to blind the judging for race, gender, etc. The proctor should be explicitly on the applicant's "side" IMO.

I used to 'sit in' on interviews, and basically say very little except to answer questions about working here or to serve as the 'linter' to get them unstuck.

You hear something very different than the person who asked the question. And in one case I discovered that my coworker had been asking a question that he thought he knew the answer to and did not; a deep clone implementation (where he never asked about cyclic graphs).

I assumed it rather than say it out loud, but my idea is to have standard, vetted questions, to reduce the risk of a question being mal-formed like that (unless you want to see how they make tradeoffs in response to an overambitious spec). Your situation sounds rather different. Do you think my scenario would work better?

The whole reason we’re in this mess is every company is trying to play moneyball. They’re all convinced that by creating a unique interview process they’ll find better talent than other companies and this make more money. The end result is more and more obtuse hiring practice which have zero practical value.

One trend I've noticed that is markedly different from when I started programming in 1998, is how dependent we've become to program via Google/StackExchange searches.

I'm not sure I know how to write anything from scratch anymore, because I just search/read/alter/test. The breadth of what I work on is 100x wider than it used to be, and so I've become absolutely dependent on quickly reading docs, copying code found online, and then deep diving into testing and rolling it out the door.

I deliver a TON more, but I'm convinced I'd bomb literally every interview I go to today. Today, for example, I updated our firewall rules, updated some Java business logic, tweaked some javascipt and PHP on the site, reversed an odd macro email-phishing virus we got, configured a new RDS server VM for clients, and updated some network GPOs. 20 years ago, I would have never worked on ALL of those things in the same week - let alone the same day.

Maybe we should interview with small take-home tasks to be submitted with some write-ups to test how people research, reason, and write-up problems, rather than writing code on the spot on a whiteboard. ...but maybe that's just me.

It's not just you :) A much more realistic way to test people is to tell them to bring their laptop, point to a problem on the whiteboard and say "here, solve it, and use whatever resources you need."

I'm somewhat like you; I do a wide variety of tasks every day and simply can't keep all the different rules and syntaxes in my head. I'm terrible at tests anyway and would probably bomb a FAANG interview badly because of my lousy memory.

As an interviewer, I would rather someone be generally smart, have a good work ethic and lots of experience solving problems. In the long run that seems more useful than being able to write computer programs on a piece of paper in a conference room.

This has actually not been my experience. I generally agree with your reasoning and have tried to run coding interviews this way, thinking that it is most similar to doing real work. Candidates can use their own computer, their own editor and plugins, whatever framework and language they prefer, can google anything they need, etc.

It causes a surprising number of problems. A lot of candidates come in with their environments just very poorly configured, run into weird dependency errors, mismatched installed versions, no linting or syntax highlighting set up in their editor because they don't actually ever code on their personal computer, etc. It's not uncommon to spend half the interview watching someone debug their dev environment, or struggle to remember the basic syntax of their favorite framework since they aren't working on the same codebase that they're used to.

I've mostly come around again at this point and decided that even if I like this type of interview best in theory, focusing on standalone problems that can be done in coderpad with no dependencies is a more reliable approach.

I use repl.it for interviews. I preconfigure the environment and sometimes fix typos if I think they're getting stuck.

A lot of developers wont use their personal computer to develop on but as a terminal.

Its actually bad practice in a lot of peoples opinion you should develop and test on an identical system hw/sw to the system you will deploy on.

Isn't that a useful way to filter out candidates that you don't want to hire? Presumably they know in advance what kind of code you're going to ask them to write, so they have time to set things up beforehand.

Yeah, you could argue that this means these people aren't that serious about us, and we should be looking for people who are dying to work for our company specifically. Those people will surely spend hours preparing specifically for our interview, and triple-check they have followed all of our preparation instructions.

The reality is that for the past few years at least in SF, hiring engineers has just gotten more and more competitive. Most people who pass our interview will have multiple other offers, often some from companies with a bigger name brand than our own. At least in my opinion, most companies in this environment can't afford to have such strict purity tests. It's supply and demand.

This balance may be changing now given the current recession, it seems even in tech things have cooled off a bit the past few months.

> A lot of candidates come in with their environments just very poorly configured, run into weird dependency errors, mismatched installed versions, no linting or syntax highlighting set up in their editor because they don't actually ever code on their personal computer, etc. It's not uncommon to spend half the interview watching someone debug their dev environment, or struggle to remember the basic syntax of their favorite framework since they aren't working on the same codebase that they're used to.

Now THAT reflects the actual daily experience of working programmers.

All the maddening little configuration options to get things working correctly, and seem to take up far more time than actually writing code to solve customers problems.

I don't even have a personal laptop, seems like bringing the employer provided laptop to interviews is not the most sensible.

I have a personal laptop but I haven't used it in a few years. I spend every productive moment deep in the weeds on my employers laptop!

I'm not sure I would even look like a proficient computer users on my personal laptop. It runs Windows, and at this point my macos work environment is so heavily customized that I'd be bound to get tripped up just by the sudden unexpected need to use a mouse.

one time I went to an interview and they had a laptop ready with something for me to do. This guy had some insanely powerful glasses to have the font so small on his computer (considering he was also quite old) and a few other things combined with it being a Windows machine and I'd been off of windows for a couple years made it almost impossible to make it work.

I'm glad my (former) employer has a scheme where after two years you can take over your "old" laptop for just €200. I'm at my new job right now with a 2017 MBP, which their current laptop budget just wouldn't even come near to. Oh and I got it revised just before leaving, the speakers were broken but there was a recall / extended warranty about that problem, it got the whole top replaced.

Wait, your employer makes you pay to get a new work laptop, or they have a scheme where you can buy your old work laptop for $200? If they make you pay to work on up-to-date hardware, that's insane.

"take over" probably means that it becomes his personal laptop.

Thought so - wanted to double check.

I once brought a laptop to an interview. Completely bombed it, because when asked to connect it to a projector I couldn't do it - I'd never used the video output on that laptop before and it was hidden behind a flip-down door that I didn't know about.

And that's it? Interviewer saw that as reason enough for you to fail the interview? Be glad you didn't get in!

More likely it's a time issue: Every minute spent fiddling with your laptop is a minute not spent on the coding problem that's being used to evaluate you.

In this case I was being asked to supply an example of a GUI I had worked on in the past, and the relevant program was installed on my laptop and there was no publicly installable version. I don't blame them at all, the problem was entirely mine.

I completely bombed an interview because they gave me a laptop but refused to supply a mouse. Plus the laptop touchpad scrolled in the opposite direction to what I was used to. I felt quite sorry for the interviewers who had to sit silent for ten minutes watching me try to copy a function from one file to another.

Mark and rikroots, I'm curious about how long it took for you to travel to the interview and back

Not long at all for me. In fact the company was next door to another place I had worked in the past.

Ok, good it was close

As the replies to this are getting at, it's really important that if you do this you don't require the candidate to use their own laptop.

You need to provide a laptop or like, a mac mini or something you know you can plug in and has baseline requirements for working on the problem you're giving them. And you need to make sure that the problem isn't complex enough that they can't do it in the allotted time if they struggle a bit with an editor they're not used to (including like, vim without their usual barrage of scripts).

Otherwise you're just trading one bias for another: winners will be people who have personal laptops set up for coding, which is not a universal trait of good employees who code.

Let them use a laptop they bring if they want, but be ready to supply them with something they can use if it doesn't work out.

Hm, maybe. I was thinking, work in an environment you're familiar with, since that's a more realistic measure.

For example, I'm an Emacs kind of guy, so if they hand me a Windows laptop, do I limp along with Notepad? or some IDE that happens to be installed? Generally, as soon as I've settled in at a new job, I've customized my work environment with something as close to Linux and Emacs as possible -- if Windows, I'll install Cygwin if allowed, and/or Emacs and GNU utilities.

It sounds like definitely an opportunity at the minimum for the existing online interviewing sites to provide a service which installs a candidate's vim configuration (or emacs, not sure how that one works) from a public Github (or other host) git repository URL. They've already gone through the trouble of sandboxing the runtimes somehow so it can't be that difficult to also sandbox the editor. Not sure how it would work for other IDEs but that should at least support vim, maybe others like VS Code and IntelliJ too.

Give me a Mac and I'll spend half of the time asking you how to do X on a Mac, keyboard, mouse, touchpad, menus and unfamiliar software.

Do you really have to test for the ability to use tools instead of the ability of reasoning? A whiteboard doesn't seem bad for that. I would like to understand how the candidates reason, how they interact with colleagues and yes, also how they handle stress.

Example from yesterday: I had to fix a problem on a software in production before the end of the day (it must be available today) and before a reasonable hour (I wanted to go out at night.) That means working fast, debug, find solutions, possibly develop workarounds that leave the system working even if the problem is still not fully understood and solved and, most important, don't panic.

> Give me a Mac and I'll spend half of the time asking you how to do X on a Mac

I worked at a place that got around this by giving the candidate a choice of their own laptop, a Windows laptop, a Mac laptop, and an Ubuntu laptop. Half a dozen different IDEs on each.

Of course, you have to find someone to take on the glamorous job of keeping track of all the laptops, and wiping them between candidates...

> Do you really have to test for the ability to use tools instead of the ability of reasoning? A whiteboard doesn't seem bad for that.

It depends: If you're a big believer in test-driven development, red-green-refactor, clear naming and good code structure, test coverage, use of version control - you don't tend to see those in whiteboard coding.

I like take home tests. Unfortunately, the one time I applied to a place that gave them, and apparently aced it, they wanted to validate it in the interview, by having me program in an environment and language I'd never used before. I get that everybody cheats at everything, but since some employers (all the ones that have ever hired me) hire without any coding demonstration, why can't you just take the risk and trust that a regular interview is enough?

It feels like that the company is lacking a programmer who is decent at communication.

It can be easily solved by asking what is a parts of code do, or how it works and why.

I mean, every other company having a giant hole in hiring policy is not a good reason for them to have it. But them not letting you use tech you're comfortable with is a stupid way to verify.

No company should trust their employees. The risk of a type II error is too great. To certain kinds of people a type II error is the worst kind of error you can make.

Why not give the take home test in advance, and then have the candidate explain their solution in the interview?

>Maybe we should interview with small take-home tasks to be submitted with some write-ups to test how people research, reason, and write-up problems, rather than writing code on the spot on a whiteboard. ...but maybe that's just me.

I'm a huge fan of giving people tasks that somewhat resemble a real world problem they'd likely encounter or at least aligns categorically with their proclaimed resume feats and more importantly on top of that giving them the same environment they're comfortable working in on a daily basis which might be on a laptop in a quiet room from 6am to 10am in the morning.

However, I'm usually instantly met with the "oh well we don't want to expect too much of our interview candidates" mentality as if asking them to do a "take home" (rather than telling them to figure out how to take a day off of work for an onsite panel) is somehow grotesquely disrespecting their time and would make the company look bad if the broader community even got a whiff that it had been considered. I'm so confused. People easily identify that white-boarding interviews suck for more than just the interviewee. And yet any time a solution is proposed that involves doing something other than putting the candidate in a room in front of a whiteboard for an hour, it's shot down.

I've concluded it's a generational thing and the problem won't be fixed until we get a wave of hiring managers that weren't schooled on the Google interview process. But, also note it's a self perpetuating issue too so :shrug:.

> However, I'm usually instantly met with the "oh well we don't want to expect too much of our interview candidates" mentality as if asking them to do a "take home" (rather than telling them to figure out how to take a day off of work for an onsite panel) is somehow grotesquely disrespecting their time and would make the company look bad if the broader community even got a whiff that it had been considered. I'm so confused. People easily identify that white-boarding interviews suck for more than just the interviewee. And yet any time a solution is proposed that involves doing something other than putting the candidate in a room in front of a whiteboard for an hour, it's shot down.

I would be (and have been) one of those people. Why? In short, likely because your take-home tasks are tedious, boring, and unrepresentative of real-life work. They are not whiteboarding, but they aren't necessarily any better.

I have done take homes as a candidate, and also have reviewed them on the hiring side. Most of the time, as a candidate, any take home I have done has been wasted time. Nothing I feel like I could count as a significant contribution, or something I could add to my resume. Take homes usually have some boring cookie cutter problem, with your most nitpicky code reviewers as the judges. Or it's something that's grotesquely oversize that's expected to be done in two hours (hedging with "but feel free to use up more time if you want"). As a reviewer, I see that it also leads to candidates doing ridiculous things like overusing decorators or metaclasses in an attempt to impress the reviewers.

Honestly, as a candidate, I would be 100% satisfied with a bitesize fix to an open source library that your company maintains, or even a bitesize fix to a closed source library under contract. Or pair the candidate with an employee who is working on a real, minor task at your company, and let the candidate "drive". Give me something real to do, not some contrived situation about an ordering system or building the next Twitter.

My mini-conspiracy theory about this is that most companies are so embarrassed about their codebases and internal process that they don't let prospective employees look at them until after they are hired.

A number of small tasks one can do in a few hours _and_ add to one's resume is probably approaching zero...

I guess portfolio is a better word I might use? Like if you ask me to write an interesting API, I could convert it into a project for future work. If you ask me to write a cool script, I could put it into an existing "interesting scripts" repo I maintain.

I have never seen a take-home project yet that I could do this with. Also they come with scary "You may not show this to anyone else, ever!" disclaimers which are more off-putting.

> Honestly, as a candidate, I would be 100% satisfied with a bitesize fix to an open source library that your company maintains, or even a bitesize fix to a closed source library under contract. Or pair the candidate with an employee who is working on a real, minor task at your company, and let the candidate "drive". Give me something real to do, not some contrived situation about an ordering system or building the next Twitter.

It can take relative ages to become familiar with a code base, especially a large one, such that you can easily hop in and commit fixes that actually work. I'm not going to spend hours trying to gain context for the code, or deploying and testing it.

The employer is going to ghost 80% of their candidates anyway, and if your fix does work, you just did free work that benefits the company for little to no gain for yourself.

> It can take relative ages to become familiar with a code base, especially a large one, such that you can easily hop in and commit fixes that actually work. I'm not going to spend hours trying to gain context for the code, or deploying and testing it.

If none of their codebases are easy to jump into, then that might be a red flag in of itself. One thing I suppose I have been lucky about is that I have worked for companies that have some (albeit very minor) open source presence. I'm not talking new features. IMO a small fix to an existing codebase can be as minor as adding a new CLI flag, or lightly modifying some existing behavior. In many cases, it's what you'd give a new intern as their first fix.

> if your fix does work, you just did free work that benefits the company for little to no gain for yourself.

Well the gain is that you learned to work with a real-life codebase. The thesis of my comment is that take-home projects do not often approximate a real life codebase well.

Interesting. I think we're getting into the implementation of take homes. My proposal is (1) that a take home technical challenge entirely replace any in-person spontaneous white board drills, and (2) that it only be issued to candidates who you're prepared to bring on-site or otherwise schedule to meet the team if they present you a working solution. I'm not advocating for them to be used as a low-pass filter and spammed to all candidates without some mutual engagement first. The expectation of the onsite is that everyone has reviewed your solution and people can discuss why you made certain decisions or how the introduction of new requirements might impact the design. Or you can talk about why you hate kubernetes or the mini pineboard cluster you built.

I disagree that the goal of an interview experience should be to necessarily give the candidate a new portfolio accomplishment. It's cool if it works out that way, for sure, but that can't be the expectation. I'm mostly interested in incrementally improving the status quo which also isn't focused around an interview day that bolsters your portfolio as an engineer.

I will say my favorite and most memorable interview experience was a phone screen followed by a take home to implement bottles and kegs (a fizz-buzz game) in javascript, a language I was/am not proficient in. Come onsite day, we worked though a ux design problem and a database design problem, we pair debugged some ruby code, and then we wrote a multiplayer back-end for my fizz-buzz client, something I'd also never done before. It definitely ticked the "add something to your portfolio" box but also wasn't stressful in the same way flash 1-hr whiteboard sessions are because I was bringing something to the table that we expanded on during the day-long session.

I like this solution a lot, but if they asked you to contribute to a public codebase, you'd be able to tell who the other interviewees were, and how many of them you were up against.

"Take home" questions are heavily biased in favor of the employer because they're not required to put anything on the line.

An asynchronous test is definitely more flexible for all parties, but it also means the employer can more forward with far more interviews than they would be able to do in person. Leading to more candidates wasting their time doing nonsense work when they have proportionally less of a chance of getting the job.

Possibly I'm just bitter because of the time I spent 4 hours on a take-home only for the company to ghost me (name-and-shame: Instacart). But at least for in-person interviews they have to buy you lunch before the ghosting can occur.

>"Take home" questions are heavily biased in favor of the employer because they're not required to put anything on the line.

Not to mention it doesn't test the collaboration aspects of real work. Imagine doing your job but you're not allowed to talk to your teammates in case of a blocker. How are we doing our jobs here?

But when it comes down to actually starting them, I will always pass if they come before any interviewing round. You don't yet have a stake in the process.

And maybe excludes short of time people taking care of kids? Maybe more often women

The "easy" fix for this is to pay the candidate for the time spent on the take-home.

That has the down side of some potentially sticky legal issues. For instance, people on visas can't do paid work for any company other than the one sponsoring them. It introduces a small annoyance at tax time, as well.

You could ask them to name an institution for you to donate money to. It would at least be something to make the world better :)

This is actually not a bad idea. As a candidate, it still gives me the assurance that they aren't just throwing out these tests to 1000 applicants.

> Possibly I'm just bitter because of the time I spent 4 hours on a take-home only for the company to ghost me (name-and-shame: Instacart).

Arthena, a Y Combinator funded company, did this to me. It truly was a waste of time.

Take home interviews should be paid, like 200$ and not require more than 4 hrs.

They should really be paid at the comp rate of the position you are hiring for (which is... a lot more than $50/hour for the Bay Area employers who have given me take-home interview problems).

Agreed. I am quite disillusioned by take home exams / online hackerrank style tests.

You got lucky. Arthena is a mess. Super disorganized and doesn't treat their people well.

So I've heard. I only went through with it because I have a good rapport with one of their employees who recently left for those reasons.

Speaking of Instacart, I scheduled an interview with them via interviewing.io, which I was told was a "technical chat," designed to get to know a little about me, and to answer any questions I might have. I had qualified for the interview by scoring well enough on phone screens on interviewing.io, so, this technical chat was supposed to be followed by a real, actual coding interview.

Guess what? It wasn't. They rejected me as soon as I de-anonymized to allow them to see my resume.

The worst experience I've had with a take home problem was when the interviewer did not even run the program I submitted. As I explained the solution it became clear that the interviewer had not even looked at or run the program. It would have been easy, it was in Python.

To top it off the interviewer then sprung a surprise leetcode problem on me that had to work and was required to be written in a language ill-suited to the task. I bombed it and the interview ended. No one ever even ran my submitted program.

If you think that's bad. I did my code submission tested the crap out of it. Then found out that the person "couldn't run it". We had a back and forth conversation about it. I even created a docker test bed with the right environment.

mvn clean package worked.

Turns out the guy was running mvn exec:java.

Worst for me was for Rakuten when they asked to basically implement a complete ETL pipeline to populate a mini-datawarehouse. It was easily more than a week's worth of 2 hours per day work that was supposed to be done within a week and worse, they didn't even bother to provide feedback.

I don't like take homes for two reasons:

1) It's less time spent talking with potential coworkers. Which for me, coworkers (and how we work together/get along) determine about 90% of how satisfied I am with my job (assuming pay/benefits are standard).

2) You're still going to require me to take a day off and stand up in front of a whiteboard anyway eventually. Every take home I've done is an early stage filter. The final round is still a whiteboard panel.

2. is the biggest thing for me. Exactly the same experience. I would be happy with take-homes if companies actually trusted them. The biggest reason I'm willing to do them is to avoid the whiteboarding.

Of course from their perspective I could be getting someone else to do it for me or something. Or maybe the take-home leaked (which becomes more probable as the company grows).

I’m surprised this is your experience.

At my current employer, that’s exactly how we approach hiring. I send the candidate a take-home that broadly covers the basic tech skills they need to be effective in their role (implement a simple test api, some questions about scaling web apps in production, and a possible architecture sketch for a described system). It’s expected to be around 4 hours of work. If they pass that, the interview is free of technical topics - we just talk about your past experience and assess whether you’d be a good fit for the team.

Is this not how other take-home assignments work?

Yes, it should be. This is how we do it

Do you feel comfortable saying who "we" is?

I too think that's how things should work if a take-home is involved, but I haven't experienced this yet. There are times I've gotten past the take-home and then had to do a technical _phone_ interview after - not even an onsite.

I've never seen it done that way -- just used as a pre-screen for an onsite that is the typical code-on-a-whiteboard scenario. Had one place that used an at home test + a one hour task in the office on a laptop they provided + several more whiteboard sessions (so long they split it over 2 days).

I can confirm this has also been my experience. I have spent multiple hours on take-home tests only to be ghosted. In no case have I ever been able to skip the whiteboard hazing because I did well on a take-home.

I'm interested to know who your employer is, if you're comfortable saying.

Not comfortable sharing my exact employer, but it’s a small biotech in the Netherlands.

How many candidates do you have doing a take-home per position?

Roughly 5-10 I’d say. We are able to weed out a good chunk in the phone screen (which is done by me or another engineer, not a recruiter)

Oh man I can relate, I once took a home project which took around 6hrs and then was subsequently invited for a full day of white-boarding with 10 different people throughout the day.

Didn't hit it off with the last interviewer so it was a no-go.

If a take-home is required I'd at least expect a code review over it and focus the rest of the interview process on soft skills rather than trivia questions that are answered essentially by the take-home exercise.

This is exactly the solution I'm imagining. You do a take home after the initial soft screen(s) but prior to coming on-site. Onsite the focus is on the work you did for the take home--discussion like "why did you model X that way?", "if a new requirement Y was added, how would it impact the design", etc. And of course all the normal soft talk and cultural stuff. Just don't put them in front of a board and ask them to solve something under pressure, please.

I had that exactly experience with a start up and it was awesome! They also were very generous in the the time given to solve it (2w).

Ten people? 10?! I guess if it's really 5 slots with 2 interviewers (though that has its own set of disadvantages, on both sides), I guess. But, that just sounds like a distressingly long day.

It was, 5 meetings including lunch. Each meeting had at least 2 interviewers and I had lunch with my prospective manager and another senior peer.

  1. 2 pms - collab and project management questions.
  2. Trivia Q&A with 2 devs and tech projects
  3. Lunch with Dir. Eng, and senior dev.
  4. Whiteboard leetcode with A researcher and a junior dev.
  5. Systems design with Dir. Eng and a senior devops
  6. Chat sessionr for questions with VP of Eng.

Needles to say I was burn out on the systems interview after having been in interviewing mode all day and the interviewers were definitely mentally drained, nitpicking on a lot of low leve elements on a systems design task.

There comes a point in an interview where you're not sure you want the job given what you've seen, and any judgement of the candidate's ability once they've checked out is useless.

You can learn a lot about a person by what questions they chose to ask you, but only if it's not a thing tacked onto the end of a long interview. If they haven't gotten much time to ask you questions, they only have your interview questions to go on. Which you know are not representative of the kind of work you do (but then, why are you asking them?)

So because you only ever talked to them about things that could readily be had off the shelf, and this is the only information they have, they can't know if you're bad at interviewing or some psycho who spends all their time reinventing wheels and then demanding that other people be impressed by your insanity.

The chances of the latter are bigger than you think. So if it's down to you or another opening, they'll take the other opening, and then you've wasted everyone's time.

> People easily identify that white-boarding interviews suck for more than just the interviewee. And yet any time a solution is proposed that involves doing something other than putting the candidate in a room in front of a whiteboard for an hour, it's shot down.

Most people take issue with the repetition of similar tests and evaluations for multiple companies. If A, B, and C were considered different stages of a technical interview, candidates are repeating A and B too often, whereas applying A and B once would suffice for several companies.

It's a problem that is realized when opportunity cost is compounded when interviewing multiple companies and it's not something that a single company alone can fix. A more "Docker-ish" solution where using one of "A" to lay the foundation for many company interviews (and not just the ones expecting you to practice Leetcode) might curtail the problem of using more time and resources than necessary.

But more than that, there is also a cognitive dissonance with the beliefs of how to reform a broken system. Programmers largely oppose industry-wide regulation while also expecting the vetting process to be more consistent and make more sense. This is a tough nut to crack because we seemingly want to have it both ways.

> Programmers largely oppose industry-wide regulation

...of programming (because the regulation would never be able to catch up to the state of the art), sure. Of HR practices? I don't think I've ever heard an engineering lead lament that their company's restrictive HR practices prevent them from testing candidates the way they want.

> and it's not something that a single company alone can fix

Sure they can: micro-credentials. Someone needs to be the CompTIA of tiny programming challenges, where you prove once-and-for-all (in a proctored situation) that you can solve FizzBuzz or whatever, and get that fact recorded in some database.

Hopefully, candidates could then put their MicroCredentialsCorp username on their resumes or into companies' application forms or what-have-you; and then first-stage HR automation (the same kind that matches resumes to keywords) could run through a bunch of RESTful predicate-endpoints checks like https://api.example.com/u/derefr/knows/fizzbuzz (which would return either a signed+timestamped blob of credential JSON, or a 404.)

Hopefully also, allow anyone to create a test (maybe have the end of that HTTP route just represent the username+repo of a GitHub repo where an executable test for the micro-skill lives?), such that these micro-credentials can live or die not by central supply, but rather by market demand.

(I think someone suggested doing this with a blockchain once, but there's really no need for that.)

Unfortunately, nobody has cracked up the problem of how to do credentials without creating a conflict of interest in favor of one of the parties (ok, maybe only in favor of the candidate). Or at least, nobody has created a way to communicate that lack of conflict of interest widely.

> Someone needs to be the CompTIA of tiny programming challenges, where you prove once-and-for-all (in a proctored situation) that you can solve FizzBuzz or whatever, and get that fact recorded in some database.

I don't think any of the major companies would accept such a solution. I mean, they make people who already passed their own interviews once re-interview if they leave and come back or apply again too long after they pass and decline. If a company isn't willing to trust its own interview process as a once-and-for-all test, why would they even consider doing so for a third party test?

This is theoretically what TripleByte is supposed to be. My experience with them was pretty horrendous though.

Heh, another thing is variable salary on a test period. Like how I was recruited out of uni, on a 6 month cheap internship barely paying a rent, with 0 commitment from the company to keep me. Long enough to evaluate me, see me contribute, teach me part of the sytem, not long enough to "exploit" me.

We could scale this to 3 months on 80% salary (whatever the company is comfortable spending on throwaway) with the full salary after the test period, with maybe even back pay and a little ceremony to make it a big deal. That would kick my ass for three months, cost me nothing long term, put a big smile in my face and let any company hire me without 15 interviews.

Isn’t that exactly what probation is for, although for full salary? To evaluate the new employee and lay them off if they don’t meet the requirements? Disagree with the discounted salary, if 20% for three months is going to break the bank, the company probably won’t be around much longer anyway.

We do take-home, and tell the candidate to not use more than 4 hours on it and deliver it any time during the next week (this is to respect their time). The testing system logs their activity, so if someone decides to spend two days on it, we'll know. Not a problem as such, but it raises our expectations of the delivery. :)

Next stage is a technical interview of 60-90 minutes where we a) try to find what their strengths are and how deep they go, and b) ask them to explain how they solved the take-home.

b) is critical. Of course it tests for the skill that is critical in teamwork: communication ability, but it also lets us catch cheaters.

After this it's HR and reference checking.

It's worked well so far.

I've been doing this for a LOOONG time. Early on, the core skill for a programmer was intensive in-depth knowledge of the language being used. We all talked a good game about what we called "reusable software," but yeah. Talk.

Now things are TOTALLY different. Doing a good job requires extensive, encyclopedic, knowledge of what's out there, how well it works, and how to integrate it. npm, nuget, maven, anaconda, we all know the drill. In a day's work it's much less necessary to implement an algorithm from memory.

What should programmers know from memory? Simple stuff like fizzbuzz. The difference between TCP and UDP. What DNS is for. If SQL data's involved the difference between JOIN and LEFT JOIN.

And, if they're experienced, they must be able to describe intelligently how their own stuff fit into the bigger system they worked on. Because there's always a bigger system.

If they're being hired to be a tech rep to an *sshole customer, possibly their performance under pressure matters a lot. Otherwise, not.

This business of high stakes quiz questions serves just one purpose: feeding the ego of the interviewer. "We have high standards!"

Yeah, so high that we wouldn't hire Jesus of Nazareth to be a storyteller.

>>This business of high stakes quiz questions serves just one purpose: feeding the ego of the interviewer. "We have high standards!"

It's scary how true this is. I know one manager who routinely rejects perfectly good candidates for making negligible mistakes in the interview. All on the grounds of "We have high standards!"

It sounds like the process works! … at preserving the candidate from your organization. :)

Haha. I worked at a place that did not start out as a software company but now had a software team (they built industrial machinery). The director of the electrical and software department was completely clueless of course. He insisted that since we are an engineering company, an engineering degree was required to work here. And that meant any candidate for a software engineering position was automatically rejected if they only had a computer science degree since it didn't have the word "engineering" in the title.

That is exactly my current employer. Our main office is in a small town, so there aren’t that many candidates to begin with (a job stays open for 6–9 months and we get maybe 15 applicants during that time)

Our software is utter trash because the business is very clannish, fragmented, and the people in charge either never ever wrote code or did it 15 years ago.

And yet we fail 95% of the technical candidates we interview because they don’t meet our “high” standards. We won’t even consider hiring remotely (but within the country because we’re making government software) even though an overwhelming majority of the company and tech team has been working from home 80–100% since March... No, we’re insisting on rejecting the handful of candidates that apply in our small town. Insane.

>Doing a good job requires extensive, encyclopedic, knowledge of what's out there

I wouldn't say that. What I feel like I've accumulated in knowledge over time is I can look at a new problem and without having the solution/encyclopedic knowledge, I can know in my gut that (a) there is a solution, and (b) the solution I'm looking at is not the solution.

Merely being able to look up things on the internet without these two things means you will either give up too early or settle on the wrong solution.

This is the most succinct way I can describe what I think >a decade of experience is worth compared to being fresh out of college.

I describe this as known-unknown vs unknown-unknown (credit to Donald Rumsfeld)

Unknown unknowns: The things that you don't know are possible and thus don't even know how to start solving. You will accidentally reinvent the wheel, most likely very badly. Or give up right away.

Known unknowns: An experienced developer most likely unconsciously knows (has read on HN/Reddit/Book/Blog some time in the last 20 years) that a thing they encounter has been solved already, at least partially.

This allows them to start searching for the answer with a few key words, because they know it's possible but don't know how it can be done.

Same comes with having a network of people who know different things, I know the bare minimum about professional pentesting, but I know a few people who I are absolute pros in the field.

> Known unknowns: An experienced developer most likely unconsciously knows (has read on HN/Reddit/Book/Blog some time in the last 20 years) that a thing they encounter has been solved already, at least partially.

I feel like my brain is full of "I read a headline about this once" tidbits, in part because of HN. It's like I don't know the answer, but I know the question has been solved before so I know there is an answer.

Yup, a lot of developers grow into generalists that know a lot of things about a wide range of subjects; they suck in a technical / algorithm interview, but they can solve most problems. They may not know all of NPM's options (for example), but they know how to quickly find an answer if they are confronted with a problem.

I honestly don't agree with this. The worst programs are written by people who know how to plug a million and one things together, but can't drill down and analyse the algorithmic implications of what they're doing. Electron runs like shit and inhales RAM is because it was programmed by people who don't have solid understanding of fundamentals. They understand a huge number of horizontal abstractions but they have no concept of how it looks vertically.

Knowing how to maximally exploit a CPU is way more important than knowing eight different Javascript frameworks if good software is your objective. And frankly, learning Node is way easier than figuring out how to structure basic, bare-bones Javascript so that it leverages your L1 cache.

And therein lies the problem. How many interviewers dock marks for iterating over columns, instead of rows? Because that matters, a huge amount. How many interviewers would give credit for "how can you speed this up?" if the interviewee said, "write it in C, and simplify the datastructures you want me to use so we maximise sequential lookups over basic arrays, to maximise cache usage." They'll look at you like you have three heads.

"Don't you know Big N complexity is the only thing that really matters if you're looking for speed?" - then you get Electron.

Performance matters when it matters. Part of the job is knowing when it matters and adjusting the approach accordingly.

Efficiency automatically matters for something like Electron because if it's not well-engineered, it will produce monstrosities like Slack.

If Slack was written in Qt with the same budget it would just be fast. Even with effort we don't have Qt apps that behave like Slack.

If slack were built in Qt it wouldn’t ever have hit the market or adapt to changes. It would be an order of magnitude more expensive too.

Do you feel that you somehow know something that nobody else knows ? It’s all about trade offs . Being able to move quicker seems more important to businesses than writing more performant software .

It's a chat client, it's really not complicated. Even if we take for granted that they genuinely had to ride on top of Electron to get to market (I think the back-end is probably much more complex), they have more than enough money at this point to rewrite from scratch.

Note that one person is literally doing that, with ripcord. Looks like shit but it illustrates the point.

It is really that complicated. They have a lot of features, integrations and platforms. I don't believe you understand how a chat client code base could be much more complex than Unreal.

Every multiplayer game on the market does 90% of what Slack does as a side note.

Can you give me one that integrates to all my business ?

Monitoring , metrics , deployment , etc.

I am also looking to play said game on all platforms ! iPad , Android , iPhone , Mac , Linux , Windows and etc.

Would be great if the chat client supported markdown as well

I understand the point you're trying to make but arguing that World of Warcraft is less complex than the Slack client is not the hill to die on.

From your posts your view of complexity seems very limited to the art of programming and making complex, fast and correct programs.

Complexity for me is the software engineering part which the Software Engineering at Google displays really well and I agree wholehearted. I would argue a game is simpler because it doesn't need frequent updates and doesn't live forever.

I work in systems at my company used in a very big scale that were written 30 years ago and are still maintained / changed. That system I would argue is much more complex than any game.

Even tho from your point of view it probably only updates files on disk :)

just here to say that vscode works plenty fine on electron and has handily beat out other IDE's you might consider better software. it's always possible to use a tool badly but it's not always the fault of the tool.

As someone who uses Emacs (which is a terrible piece of software in its own right), looking at the top feature requests for VS Code is like looking into the twilight zone. It lacks absurdly basic functionality.

Granted, they made Electron run fast enough to be usable but I wouldn't call it good software.

It might lack functionality that you use on a daily basis, but I don't think it lacks any objectively basic functionality. In my opinion this is proven by the fact that thousands (millions?) of programmers, including me, work with VS Code on a daily basis.

What's the alternative? All editors are kind of crap. Compare them to raw level of engineering you find in, say, a video game. It's a different world.

Imagine VS Code was exactly how it is now, but had the extensibility of Emacs. It would blow everything else out of the water. Why isn't it like that?

> Why isn't it like that?

Because most of us don't care. I need my code editor to just work out of the box. I want to be able to write code and install a few plugins to enhance the tools I'm using. Hacking my editor is a waste of my time and even if I wanted to, I'm certainly not going to learn lisp to do it.

Sublime Text is nice if you care about good engineering and performance.

The extensibility of emacs is insane is why, it really is more like an OS that happens to have editor functionality. I think the only thing that even gets close is Eclipse which has a ton of performance issues which I'm convinced are due to the (enterprise-y) way they try to achieve that extensibility. After that would be the IntelliJ platform, it's usually much faster and nicer to use but also has performance issues and is known for using a ton of RAM.

VS Code is doing well here compared to its competition. It is quite extensible, you can add extensions to it written in JavaScript that can add new UI and make changes to the editor view. It's also much more performant than either of the other two IDEs I mentioned, even after you load it up with extensions to replicate some of the most used features from those IDEs.

This might be off topic, but what basic functionality does Emacs have that VS code doesn't? I know there's SLIME and Org, but I don't think you meant that.

Difficult to answer directly, but scanning through the most popular feature requests, most of the non-VSCode-specific feature requests have been solved (somewhere) by Emacs:

https://github.com/Microsoft/vscode/issues?q=is%3Aopen+is%3A...

I will say that this was way worse a couple of years ago. They seem to have added a lot since then.

your definition of good software here reminds me of the critic in Ratatouille who loves food so much he doesn't eat any of it. must be terrible living in a world where everything sucks as badly as you perceive it.

Hacker News, which is literally just text, can't display every comment in this post. There's a disclaimer at the top. Scroll up and read it.

How much money does the Y Combinator Web team receive? Do you think its enough that they should be able to render... Text?

> Knowing how to maximally exploit a CPU is way more important than knowing eight different Javascript frameworks if good software is your objective

Which is why 90% of all software is very profitable crap.

> Electron runs like shit

Does it? I've seen some electron apps that run like dogs, and others that seem perfectly performant. So, I'm curious... in what way does the framework run like shit? In what way could it be improved?

> How many interviewers dock marks for iterating over columns, instead of rows? Because that matters, a huge amount.

I think you mean the difference between SoA and AoS. I'm not sure that one is inherently better than the other, except the former scales well for large homogenous datasets.

I doubt optimising for DSPs/SIMD/big-data to maximize throughput and reduce cache-misses, is something the typical FAANG employee needs to know about. Could be wrong.

> "Don't you know Big N complexity is the only thing that really matters if you're looking for speed?"

It does, even if you cache-align and pool your data. And you can optimise your in-memory data-structures (and database IO) later, if profiling finds them to be a bottleneck.

>I've seen some electron apps that run like dogs

And almost no Qt apps run like that. "As long as I don't notice it most of the time, it's not slow." No, sorry, I don't agree. Frameworks should be fast. Slack is unforgivably slow.

>I think you mean the difference between SoA and AoS.

I mean literally in the interview when they do a nested for loop over primitives. Do they lose marks for going row-first? Why isn't that considered a basic rule? It's not complicated. Most interviewers don't even realise there's a difference (!!).

>I doubt optimising for DSPs/SIMD/big-data to maximize throughput and reduce cache-misses, is something the typical FAANG employee needs to know about.

Twitter is so bad it crashes my browser. It's 90% text! The only ram-hogs are auto-playing videos that dissapear after you scroll past them. These engineers are paid half a million per year. Half the time the website doesn't even load (!!!). That's insane.

>And you can optimise your in-memory data-structures (and database IO) later

Ehh, skeptical. Unless your data & algorithms are structured as contiguous arrays of primitives to begin with you're going to have trouble refactoring the abstraction.

FWIW I work in a place which uses a lot of C++ and cares a lot about performance.

I have seen a lot of really bad Qt UIs as far as responsiviness. The Signal/SLot framework can get really hard to follow in huge code bases with lots of state. The whole thread interaction in Qt is not ideal.

It is interesting because most of people are now actually using PyQT or pyside. Even with these being slower than pure Qt. Because it is easier to adjust to business requirements with Python.

Oh, and I have personally rewritten some of these UIs in React and replaced them with a web browser which made them much more responsive.

> Half the time the website doesn't even load (!!!). That's insane.

I’d assumed Twitter showing me “an error occurred” on about 50-75% of inbound link-follows to their site, across multiple browsers on multiple operating systems, was an intentional “feature” to drive me to their app.

It's a deliberate business tradeoff.

> Do they lose marks for going row-first?

It's not objectively wrong to do this. Suggesting that "SoA is always right", tells me that maybe you don't understand the trade-offs between expediency, readability and performance... or understand the trap of early optimisation. Always measure first before optimising, otherwise write code that's easier to read, and simpler to write.

> Twitter is so bad it crashes my browser. It's 90% text!

Not sure how knowing how to micro-optimise cache-effects fixes these issues. Profiling and spending time on perf does, but they've decided it's just not worth their time.

> Ehh, skeptical. Unless your data & algorithms are structured as contiguous arrays of primitives to begin with you're going to have trouble refactoring the abstraction.

The keyword is refactor. Developers and organizations do it all the time - or at least they ought to.

Disclaimer: Used to work in the video games industry. Literally worked for years doing perf, and cache-level optimisation.

You're responding to things I didn't say.

No, I don't mean SoA, I just said that.

I don't think twitter is crappy because of cache misses, I think it's crappy because they don't have a solid understanding of what their code is doing. Ignorance of caching is another symptom.

If you build your game on an entity-component system, you can't "just refractor" that to make it contiguous. Your structure is pretty baked.

> I don't think twitter is crappy because of cache misses, I think it's crappy because they don't have a solid understanding of what their code is doing.

It's my hunch that you have little idea of what the code-bases for twitter are like, nor what factors are at play that affect usability issues.

> If you build your game on an entity-component system, you can't "just refractor" that to make it contiguous.

Moving to object pooling of components is very do-able. There might be some extra work, like extracting a generic matrix hierarchy and physics primitives out, but you'd only do such a thing if you found that d-cache misses on v-table lookups, or if i-cache misses due to update code churning were of real concern... and these could realistically be mitigated with homogenised object pools for the particular use case... though usually yes.

Edit: I'd expect anyone on an engine team, or technical programmers at a game dev shop, or maybe even someone working on a browser renderer to have such knowledge from day 0, rather than the average FAANG employee.

> Electron runs like shit and inhales RAM is because it was programmed by people who don't have solid understanding of fundamentals.

I feel like this is a huge oversimplification. I’d argue that for sufficiently large projects (say, Chromium) every single programmer who works on it could have a strong understanding of performance fundamentals but the product itself could still have performance problems because there are necessary levels of abstraction involved and no one person has the entire codebase in their head. Performance problems could come from emergent behavior that could not have been predicted by the original engineers.

But, as for implementing an app on top of Electron. Say I need to write a cross-platform app with a consistent style and fluid animations. I know how to write a NEON implementation for ChaCha20 for fun, or write a zero-copy binary property list parser that’s faster than anything I’ve found. I have a decent understanding of performance. That doesn’t help me write this cross-platform app. I don’t have to time to write a novel native UI toolkit that allows me to share code while maintaining platform consistency and native performance. I don’t have time to fix the memory consumption of Chromium. I’m going to write my Electron app in JavaScript, carefully, and hope the performance will be acceptable (but not great). I don’t see how that can reflect on any individual programmer poorly.

I disagree. As evidence: games are way more complex, have enormous teams, and manage to simulate an entire game world rendering render hundreds of thousands of polygons 60 times every second. Twitter crashes my browser.

That's because GPUs are insane and purpose built. For an honest benchmark, you'd only look at instructions executed on the CPU which won't be any rendering, but stuff like world ops, AI, game flow. Some of those are not really that taxing.

As an example, Stellaris had performance issues that were CPU bound for a while and it's gotten better. It's not a visually demanding game after all.

So run Electron like a game. Leverage the GPU, if that really is the issue. Or run it like Qt, which manages to avoid that problem.

I just think that's a weak excuse, especially given most games have tremendous complexity running through the CPU too.

(As a side note I don't think paradox games are a good example. They're famously poorly optimised.)

But I don't think that it has _anything_ to do with the individual programmers inherently understanding performance better.

There is likely a huge organizational focus on performance and more people expressly dedicated to the task because it is a higher priority for Epic. It is directly correlated to Epic's ability to deliver a successful product to game companies, whereas for browsers it is slightly more orthogonal to their success.

LLVM has a bunch of contributors who likely understand the intricacies of machine code better than either of us, and:

> Each LLVM release is a few percent slower than the last.

> The larger problem is that LLVM simply does not track compile-time regressions. While LNT tracks run-time performance over time, the same is not being done for compile-time or memory usage. The end result is that patches introduce unintentional compile-time regressions that go unnoticed, and can no longer be easily identified by the time the next release rolls out.

https://nikic.github.io/2020/05/10/Make-LLVM-fast-again.html

Electron is a running joke because of how horrible the performance is. Again, I don't buy it. I think it's much more a matter of "they can't" than "they don't want to."

I think if those LLVM contributors understood compilers better, they wouldn't be introducing those issues. Look at Jon Blow's achievements on Jai. LLVM is the bottleneck for his entire compiler at the moment, and he's planning to replace it with what... Three people or something? But I also think there are structural issues there. I don't think a top-down project from a large company gets to use that excuse.

Think about the requirements for market success of the different pieces of software. Games most run fast and any performance you safe can go back into better graphics. Performance is crucial. Electron is hugely successful despite not being very performant.

Developers love thinking about performance. It's fun and asked us to use so many of our skills. However, for many products other quality aspects are more important, like maintainability, extendability and expediency of development.

That performance often falls by the wayside on terms of business priorities ends up being reflected in the typical developer skill set.

I don't know if I agree. Its plausible, but also these businesses are monopolies, so even if they churn out shit it doesn't undermine their market position.

But I think Electron in particular is a symptom of the fact that GUI frameworks are in general horrible. There was a market gap for usability because using everything else was obtuse, and now we're paying for it.

Which businesses are monopolies? Slack surely isn't. If there's a market for faster chat clients because that's what people want then looks like a business opportunity?

FWIW all of slack competitors are also electron (rocket.chat et all), so it seems like you possibly know something that other engineers don't and might have a competitive advantage!

Google is a monopoly.

If someone released a GPU-accelerated version of Electron that had the performance of the GUI in basically every video game, it would destroy Electron. Now, would it be popular? I'm not sure, but let's say the answer is no. Ok, now imagine Google released that.

The likelihood that people who created Electron don't understand caches or cache locality is close to zero. That's an oversimplification. Electron is a large project that does many things… there are many reasons why it could have been slow. Today there are many Electron based apps that are quite decent.

> Knowing how to maximally exploit a CPU is way more important than knowing eight different Javascript frameworks if good software is your objective. And frankly, learning Node is way easier than figuring out how to structure basic, bare-bones Javascript so that it leverages your L1 cache.

Learning Node isn't easier than learning how to structure basic, bare-bones code that leverages L1 cache. In fact, it's quite the opposite. You can learn cache locality as a concept in much less time than you would learn Node (or any programming ecosystem for that matter).

I don't buy it. Unreal Engine is a larger project that does many more things, and the speed at which it does them compared to Electron is not even in the same universe.

How can you compare unreal engine with electron ?! That makes no sense at all.

The fact that people think they're somehow fundamentally incomparable is the problem.

Software is all about trade offs. I would actually recommend you the book Software Engineer at Google since you mentioned them so many times: https://www.amazon.ca/Software-Engineering-Google-Lessons-Pr...

The first chapter is all about that and how software engineer isn't programming. You seem to think as programming and I believe you're missing the bigger picture.

I understand what you're getting at but I think you're seriously underestimating the implications of what I said.

Seriously. This whole part of the comment tree has been fabulously enlightening. Great insight into the psychology of how we end up with chat clients that do less with 20x the resources than entire operating system + office suites of yesteryear.

I feel you tried to be snarky and I didn't really follow even what you tried to imply! Regardless would like to have your opinion if you don't mind explaining it better.

I’d figured it was mostly cheap companies driving the broadly terrible performance of modern software for often fairly small benefits to speed and dev cost, but it turns out there’s a much stronger contingent of software developers with not just a tolerance for business-driven trade-offs, but a strongly enabling attitude toward the whole thing, apparently not seeing what they’re doing the same way I do at all. Adding this information, observed software quality makes a lot more sense to me now.

Software developers are hired by business to solve business problems.

Optimization of speed is sometimes it, sometimes it isn't.

I have worked accelerating algorithms in VHDL with a PCI-e interface, embedded linux without a MMU because a MMU uses too many logical gates, digital signal processing systems (FFT, goertzel, sigma delta filters) that had to process a lot of data under uS and etc.

Now a days I work more in the devops space, full stack dev and whatnot. I have worked with a lot of technologies, different constraints, different teams, different companies (12 in total) in different industries.

Trying to paint all business and developers as bad or as cheap because business requirements do not align with your view isn't really fair.

> Trying to paint all business and developers as bad or as cheap because business requirements do not align with your view isn't really fair.

Yes, that would be unfair.

Twitter failing to load on a stable connection and crashing the browser if you scroll too far is not the result of business tradeoffs. It's a failure to achieve their own goals.

Are you sure there's not something wrong with your environment? Twitter has lots of users (me included) and is not a problem I have seen in any platform I access twitter from.

I also hit errors following links to twitter more often than not, reliably, on all sorts of environments. It feels like they’re doing some kind of cache-related routing based on headers. Often “refresh” doesn’t fix it, but hitting enter in the address bar does. Seen on iOS, macOS, Windows, and Linux, mostly in very normal browsers (I’d expect mobile Safari in particular is at least on their top-5 most-seen browsers). I’d assumed they were trying to annoy me enough that I install their app. I’ve seen many others report the same kind of reliable, very frequent errors.

Notably, I’m also not logged in, which might be another thing they’re trying to get me to do and another difference between those who hit this more often than not, and those who rarely or never see it. It’s been like this for years.

That sucks! I heard reddit also does things differently when you're logged in/ logged off. I haven't had a bad experience with these platforms myself but it sucks that it is so bad for other people.

The page loading thing used to happen to me regardless of login status, but it's less frequent now. The crashing was mostly just on mobile - that's still a problem.

If I remember right the app would also crash but it could take bigger loads. Not sure, I uninstalled it.

I'm not sure it's the cheap companies or developers driving this. Even companies that pay really well produce pretty slow software (Gmail, etc.). It's mostly the ecosystem in which software is developed.

As someone else said, businesses hire software engineers to solve business problems. Speed often isn't a priority. Fast software will make the experience perceivably better, but it's hard to get on a sales call and tell a potential customer that your application loads much faster when they are looking for feature X. And most companies buying software are looking at a checklist of features first and then maybe the experience.

You also need to put in the time to make software fast and then keep it fast. It's not something you can tack on at the end—if you want something to be fast, that has to be carefully considered from the very beginning. By the time people notice something is too slow, you're burdened by half baked architecture and a monstrosity of a codebase. At that point, it's near impossible to really make anything fast.

> most companies buying software are looking at a checklist of features first and then maybe the experience.

I think the hugely successful SalesForce proves this point. Just one standout among many such products.

I would argue they won because of marketing, not because they had unique functionality. Features are not that hard to build, most basic tooling can be cloned with a relatively small budget.

Most enterprise businesses win because of better sales and distribution, but your product ultimately has to support your sales. Salesforce had both (and other advantages). How does sales work in medium to large B2B? You'll get a RFP with a list of questions, you'll often get a list of features that company X wants. Naturally, as a company, you'd focus on building those features instead of optimising speed.

If the market values fast software, you'll see a lot more fast software. There _is_ a lot of unnecessarily slow software these days, so it _is_ starting to happen. Think Superhuman, Linear.app, etc. — an entire category of software based primarily on the idea of being fast, aimed at the power user. If these apps succeed, you'll see a lot more of them. If the market doesn't value fast software, most commercial software will be slow.

I don't really agree with the premise. Sales by definition exploits the fact that the consumer can't differentiate between products on their own. That leaves a lot of leeway for broken software and misleading marketing.

I don't think companies choose to make bad software as a tradeoff. Salesforce had enough budget to build out those features and not be terrible. I think in general, when software is bad it's because the company wasn't able to make it good. And I think the distinction between those categories is important.

SalesForce ticks enough checkboxes. Competing products may tick the boxes too, but without great marketing that won’t matter, the company that can sell their product will win.

My point was that SalesForce has fairly poor user experience and performance, but does offer the features. It has the additional advantage (for SalesForce) or quick lock-in once the customer commits to it. SalesForce has improved the user experience over time because they have the customer base and revenue to allow that.

This. Last interview I bombed because no one would ever remember what they were asking for in the interview test. I said I'd just google it and then dropped "Never memorize something that you can look up." ― Albert Einstein

I like this quote, but I think the point comes across better if you rephrase it:

> Never _intentionally_ memorize something that you can look up.

I guess you can't quote Einstein, then.

https://duckduckgo.com/?q=einstein+memorization+quote&ia=web

Yup, the tasks that you repeat often enough will get stuck in memory.

In theory anyway, I always have to look up how to remove a remote git branch or how to do the euro sign on a Mac.

Why would I trust my memory on how a utility works with an empty set when I can just look it up?

So many languages, frameworks, and libraries copy the same functionality and every single one does it differently.

The cost of looking it up is a little bit of humility. The cost of not doing so is a small chance of spectacularly breaking something for my entire team. Suck it up, apply the Precautionary Principle, and most importantly, get over yourself.

I have a standard preface to start a live coding or technical session with a candidate to account for this. I look up stuff all the time, and expecting candidates to not do so is cruel.

> We're going to go through a few technical exercises together. It's open book, open documentation, use whatever resources you would normally use — though please do not copy and paste complete solutions from StackOverflow and guess&check to reach our solution. It's open language, so please select whatever you're strongest in. If it is a language I am not familiar with, I will ask questions. Some of those questions may be naive. Use whatever editor and environment you're most comfortable in. Please ask questions of me. There is no specifically correct answer, though your solutions should hit a level of baseline correctness. These problems are designed to mirror some typical day-to-day problems you might face on the job; we're not looking to test rote memorization of _Hacking the Coding Interview_. We want working code, but some aspects or edge cases may remain in pseudocode as we iterate on these problems together. There are no designed 'gotcha' moments in these exercises, and if you feel there may be a 'gotcha' moment, ask clarifying questions. I would rather you ask questions than struggle through a solution.

It works well for modern development.

Maybe I should start saying (as a candidate):

> I understand you want to test my programming ability, and in order to do that, we should do it under conditions as close to real world as possible. Because I don't know every algorithm in existence, I'm going to use the internet like I do normally to look stuff up. And, I'm going to use my own laptop that I've brought here. Is that cool?

Not really, of course, but I'd like to.

Yes, you should be saying exactly that.

Just do it ahead of time instead of on the spot. You're probably emailing back and forth with a recruiter. Send them the exact comment you posted here, and ask that they make sure that your interviewers know about it.

It's amazing how little things can throw you off your game. I had a couple where they handed me a MacBook set up for "traditional" touchpad scrolling (where it works like a scrollbar instead of like a touchscreen).

Every time I tried to scroll it went the wrong direction!

Of course in an interview we aim to please, so I lived with it for a few minutes, but finally asked "Do you mind if I change your touchpad settings?" Of course that led to a brief discussion on the virtues of traditional scrolling.

This was not only precious time wasted, but a minor blow to my self-confidence. Imagine you tried to drive a car or fly an airplane and they had sneakily reversed the steering wheel or yoke and pedals! You would be lucky to survive.

In fact this is why the preflight checklist for a light plane always includes "controls free and correct".

Most of my job depends on analyzing the runtime complexity and memory complexity. It's my passion. I don't memorize any language, but boy do I understand the small complexities and seemingly trivial interactions that have profound implications. When I interview, I ask them about what they enjoy about programming and how they approach problems that they're stuck on.

I believe we should program the way you do nowadays. This is what people praised in the past so much as code reuse. This is what research people call standing on the shoulder of giants. We should reuse other's wisdom (well, you need to filter what wisdom is and what not first, of course, which is not that straightforward itself). Most profession work on pre-manufactured components for the sake of... well, many things (its a story of its own)!

Do it smartly, know what that component is, if that is good for the purpose, if that is the best for the purpose, its trade-offs, but that should be the primary way. With fallback to reinventing the wheel.

We still will have to provide custom built solutions from scratch from time to time (in fact in a lot of cases, the CS is still not mature enough for proper higher level operations) but better to rely on tested and ready pieces.

Recruiters seems to have no clue how to select people for the particular position therefore they go back to the ancient times of CS like this dumb whiteboard thing. I also felt like they seems to expect that custom made employees exists out there and they only need to wait for the perfect candidate, not really expecting someone to learn while the continuous learning is the industry norm, the inherent requirement. I've seen positions remained unfilled for more than 6 months sending away partially fitting experienced candidates.

I failed at technical interview where you must react instantly and accurately without any help normally available in everyday life (Google). After looong time of programming and learning various approaches I find myself question myself continuously even in fundamental matters. Or is it something by age? Nevertheless while being young, self confident and energetic and the Google style of programming was far from invented my fingers were spitting huge amount of brilliant code that did not work until I fixed all the stupid mistakes I made, which took frustrating amount of time. Coding gives a good amount of humility by showing you that you are to blame in every last cases. Every time! Making mistake, misunderstanding something, not knowing necessary elements, misjudging aspects, not checking the reliability of some component you rely upon, it always comes back to you as the computer will do exactly what you tell it to do, it will always be you. Frustrating. I learned to doubt myself and verify seemingly evident matters which does not play well in whiteboard interviews as you might imagine (in fact once I had mental whiteboard interview, speaking about solutions, how common is oral programming in practice?). This is an unnatural and sinfully inaccurate way of judging how one will work in a position. Due to the constantly changing (improving?) profession and the huge amount of beautiful aspects it entails keeping every bit of detail in head that you can spit out in a second notice is far from being a smart expectation. It happened more than once that the right answer occurred me after walking away from the interview as the knowledge bubbled up of the sea of experiences. Once I told someone that sorry, I never met that technique before just to realize later it was an essential component in a task several years back, I used that quite a lot for that one task.

I did not change position too much (3-8 years spent in one place) but I never been hired based on whiteboard test but by trial tasks or probation period, fulfilling the need of the position (leaving due to dissatisfaction every time). In fact I just met whiteboarding recently and I find it hugely inadequate to judge competency.

> Today, for example, I updated our firewall rules, updated some Java business logic, tweaked some javascipt and PHP on the site, reversed an odd macro email-phishing virus we got, configured a new RDS server VM for clients, and updated some network GPOs. 20 years ago, I would have never worked on ALL of those things in the same week - let alone the same day.

If you're happy, then awesome, but the reason you would have never worked on all of those things before is because companies are loading up more responsibilities onto technical people to cut down on costs.

You carry the skills of 3 roles from 10 years ago: Sysadmin, developer, and reverse engineer, and yet you're likely only being paid for one.

Unless they're getting 120 hours of productivity out of him, then no - they're just more in need of fewer generalists than many specialists. Specialization is great, but only if all of the other necessities get covered.

I only used clever algorithms maybe, MAYBE a dozen times in 12 years as a java developer.

I know I can, in university I implemented huffman compression(I know there is nothing special about it algorithm-wise) in java and I got the best run-time in 2 series of ~100 students, and by that I mean orders of magnitude better; my runtime was on par with the best C++ programs I could find, and my program could work on arbitrary file size, the fastest C++ program I could find at that time (20 years ago?) loaded everything in-memory. I wrote all the data structures from scratch, using java structures consumed a lot of memory and was way slower.

But the jobs I got in my country as a J2EE developer were much better paying, at that time at least, than any C++/algorithms stuff.

If I was younger, with the same singlemindedness I used to have, I'd absolutely spend 1-2 years tuning my algorithms skills so I could get those sweet FANG job.

Or if I had the ego of my ex submediocre-redneck boss who thought Spring Framework was bullshit, without actually knowing it, and got into programming competitions(only to be butthurt by his results because he did not put in the effort to actually practice algorithms and data structures), again, I'd study hard on algorithms and datastructures.

Or if that was my hobbie -- algorithms and datastructures is 100% a valid hobbie -- but it is simply not my hobbie.

Sure, if I encounter a hard problem, I can look up the best algorithms, copy&paste, modify, optimize and optimize untill I'm satisfied with the result, but there simply is no payoff for me to be an algorithms grand master.

This makes me very nervous - I agree it is technically possible to do that volume of change in a short amount of time now, but I don't think it is possible while still accounting for QA / testing and documentation. That is a massive amount of change in a small time without any team oversight

i worked at a company where i had done over 150 interviews and the company did at least 10 interviews a day at the office i was at. we did metrics on what times of day or days of the week or rooms.

before lunch or 4pm or later interviews we’re worse across all candidates

certain rooms we knew were smelly, loud, or poorly converted from like a storage closet scored worse across all candidates

we took some rooms out of the rotation and made sure everyone was well fed/had drinks. the effect is real

Oh no, confusing median vs mean is a very specific interview question I've bombed in the past so hard that the interview almost immediately stopped!

So is the moral of the story that this hire worked out or not? Was this a good coder?

That's another downside of the system: learning that is too much effort, unless they land in your team. But knowing how we hire only obviously overqualified people, I trust he did well.

> He also got hired - apparently the committee agreed that "can't think when hungry" is not really that important a flaw.

I hope you suggested that this employee get tested for hypoglycemia. It'd probably greatly improve their quality-of-life to go from "my brain isn't working and I don't know why" to "my brain isn't working and I know exactly why; I need to drink some fruit juice."

Hah. "Did a tech interview, found out I had diabetes."

> Couldn't tell an average from a median.

Shouldn't that be tell a mean from a median?

It should. There is a sibling thread about this ;)

(This subthread was originally a child of https://news.ycombinator.com/item?id=23848916)

It seems pretty obvious to me that this kind of interviews are simply there to assess how much you want to work for a certain company. As in: would you be willing to study for months, go through mock interviews, read books, test your skills etc. to have a shot at working for us? Even though the majority of that stuff will likely have zero impact on your day to day work?

That's all there is, really. I've had no problems in the past implementing complex algorithms, if I needed it to get the job done. Today I'd barely be able to tell you their names. But to get to the point of knowing most of them by heart and being able to implement them with no help whatsoever, would take a lot more work and dedication, knowing that you'll probably end up not retaining most of it anyway.

The same system is used in competitive sports and academic tests. They mostly measure how much you really want to be part of that league/team/school, rather than whether you'll do well or not. If there was a perfect correlation between pre and post entrance test performance there wouldn't be PIPs, performance based layoffs etc.

Really? I didn’t study at all for the interview for my current job (although I did put up with some very broken HR software for a few weeks while getting hired.)

The funny thing to me is that Google is known for this sort of thing, they also created a product that makes the need to memorize all this stuff obsolete.

Here's the truth that nobody wants to believe: you will never know how good someone you've never seen perform will perform until they actually perform.

>for a certain company

But the same skill is transferable across arbitrarily many companies. This is why many prefer interviews to take-home tests, so that they do not invest too much time in any particular company.

You're missing the quality they're actually testing for: ability to study and dedicate yourself to a task. Or in other words work ethic. The algorithms don't matter but the fact that you can spend months learning them is an important skill. Intelligence is less important for success than work ethic which is why work ethic is tested for so much. Specifically, work ethic and ability to learn in a related domain as, for example, literature study skills may not transfer.

>If there was a perfect correlation between pre and post entrance test performance there wouldn't be PIPs, performance based layoffs etc.

Nothing is perfect, saying it's useless because it's not perfect is a silly argument.

In my experience, certainly no interviewers think that they're screening for "ability to study and dedicate yourself to a task." In being on both sides of the table for more than thirty years, I have never heard an interviewer mention that.

Companies encourage you to study for the interview, providing you with links to books, guides and preparation material. They'll delay the interview to give you time to study more. So it seems pretty clear to me that corporate management sees it as a test of study skills even if interviewers don't.

edit: I'm talking about large tech companies, not the startups blindly copying their policies with little thought or consideration.

Never seen that happening. Have interviewed with a few FANG companies. There are hundreds of things they can ask and is not reasonable to study for them all

I've had FAANG tell me to read Cracking the Coding Interview, sent me links to leetcode/hackerank, sent me PDFs with notes on what topics would be covered for domain specific interviews, linked me to their blog posts on architecture design, videos by their engineers on how to prepare, given me the literal behavioral questions I got asked, etc, etc. Facebook sent me well over a dozen links to guides for my interview and half of those had further links in them (including to a list of practice questions on hackerank).

edit: It is also very reasonable to study having done it myself and talked to people who have gotten into these companies. At 15 minutes a pop you can do 100+ leetcode questions a week. Spend a few months and you'll have studied almost every reasonable question. Most of them are repetitive patterns but around 5% have tricks you need to memorize. That covers 2-3 out of your 5 interviews. Behavioral you need to memorize around 5 questions that they'll basically give you ahead of time. System design is harder but there's videos for the 10 topics that will cover 90% of the potential questions.

I really think having people pass these coding interviews is so that the interviewer isn’t responsible in selecting the wrong person.

What kind of team are you with? And how many leetcode-style problems have you solved in your actual work over the past week?

I went through the same thing for my Facebook interview. How do you do this if you have a day job or even worse, a day job and a family?

I feel these companies prefer to promote internally so losing senior talent doesn't bother them as much. Fairness is not a quality I associate with them.

That is definitely not true any more, as they got bigger most of them desperately tried to hire experienced external hires, on the principle that nobody gets for fired for hiring an ex-Googler.

Can you post the source of system design videos ?

> At 15 minutes a pop you can do 100+ leetcode questions a week.

By that logic, most people would have completed all 1500 LC's in 4 months.

Numbers say otherwise. Each medium difficulty problem takes an hour to solve, assuming you can come up with a non-brute-force solution. Most of them force you to lookup, understand and implement other people's ideas.

If you can do 3 problems a day, completely on your own without any lookup, you're doing good pace.

Some companies even go further. I invited for McKinsey as an experienced hire. They not only sent me study materials, but also setup phone calls with McKinsey alums to practice for the interview and gave me the flexibility to speed up or slow down the process based on how prepared I felt for their interview process.

Everyone involved in the process wants you to succeed. If not out of the goodness of our hearts, then because we've invested time. Under-preparedness is a stupid and preventable reason to lose a candidate.

When someone comes in and bombs, that is at least valuable information: we don't want to work with this person. When someone comes in and seems smart and capable, but makes hardly any progress on the problem because it's obviously their first time on a whiteboard, it's just regrettable all around.

We're not selecting for preparedness. In fact, we try to select against it: interviewers take pride in designing twists that separate understanding from recall. If you seem to be recalling, we're supposed to pick a different question, or failing that, report a null signal. But we do need you to be prepared enough that your interview skills don't get in the way of your actual skills.

I've always thought the SAT is basically gauging the same skill: can you study for a test and do well? The grades you get in college are mostly determined by how well you can study for tests. In the process, hopefully you learn the subject, but that always felt like a side goal.

I think tech interviewing is somewhat similar: are you focused and dedicated enough to learn something arbitrary and produce code for it?

The LSAT too. It's just a giant test-game. By the end of studying for it, I was reminded of how I used to approach StarCraft. Once you figure it out and spend enough time on practice, scoring in the top percentile isn't very remarkable. Nothing you learn will help you in law school but you will have proven your ability to study. (Rare exception: I had a friend who didn't study and just took the test and scored in the 97th percentile. This is very uncommon and I am just taking him at his word that he didn't study.)

> The algorithms don't matter but the fact that you can spend months learning them is an important skill.

And you're missing what these tests are actually doing instead: Figuring out which people have already seen the problem at hand, and probably memorized all/some of it.

The problem is that these interviewers will not pass you if you don't solve the problem almost perfectly in an amount of time that is way too small for someone who has never seen it before.

To make matters worse, many of the solutions require some trick or mathematical trivia to get right. Does that really prove you know your Algos & DS?

>And you're missing what these tests are actually doing instead: Figuring out which people have already seen the problem at hand, and probably memorized all/some of it.

That is literally my point. They are very much testing your ability to study for months on end. Study as in memorize. That is by design at this point and not a side-effect. If you aren't willing to show the ability to study successfully for months on end then they're not interested.

edit: Over the course of months of grinding leetcode you will see 99% of the problems they are going to ask. Then it's about memorizing the tricks and patterns.

edit2: And as I said below their recruiters are very open about what you need to do (grind leetcode) to succeed at the interview.

I feel like that selects against innovative people. Memorizing problems that have already been solved is basically the definition of re-inventing the wheel.

I'd argue that memorizing solutions is the exact opposite of reinventing the wheel. You implement an existing tried and true approach rather than trying to come up with a clever solution yourself. Most of the work at large tech companies isn't innovative but rather it's tedious work that has to be done to high standards with verbose documentation. This seems to select the right people for that. Remember that selecting against doesn't mean excluding. Enough innovative people will get in to keep pushing things forward.

You couldn't be more wrong. Memorization is one of the lowest forms of intelligence. Just because you can spit out a DFS algorithm, it doesn't mean you understand why and how it works.

FAANG employers want innovators who think outside the box. You can't do that with sheer memorization. Nothing new would be developed if tech companies hired people who just worked with what they already knew.

There's a difference between software engineers and coders. If tech companies didn't want engineers, they would simply hire coders from a coding boot camp.

> You're missing the quality they're actually testing for: ability to study and dedicate yourself to a task. Or in other words work ethic

TIL I've been "hacking" all their "tests" :D acing them just by curiosity & interest in computers in general, a bit of intelligence and a lot of laziness!

FAANG or other companies? FAANG interviews tend to be timed enough that even if you're intelligent enough to come up with a solution you won't have enough time if you haven't studied. On average you'll get roughly 17 minutes to solve, explain and code a leetcode medium with optimal efficiency and no bugs. Either that or 35 minutes to do a leetcode easy followed by a hard. Again with optimal efficiency and no bugs.

Other companies. Yeah FAANG is probably trickier. Part of the reason I haven’t ever applied is that I cannot be bothered to study. In addition, their salaries aren’t that overwhelming in Europe so why bother...

The stock grants are very, very good, especially in an insane market like we have right now.

Ethics? You are asking people to study and spend hours working on LeetCode problems for something close to completely useless. On the ethical scale, people that accept to spend hours of their limited life doing so are probably not the ones I would think of as "looking to work on something great"

Most of the jobs at these companies are boring grunt work and definitely not working on something great. So they're selecting the right people for that. You do get paid significantly more than other companies so there's a very real incentive for playing the game.

Yeah writing an OS, a machine learning framework, designing deduplication for a photos app with a billion users , working on on board sensors for a self driving car. Such "grunt work". I'd love to know what exciting and hot thing you're working on ? Is CRUD the new sexy ?

You're looking at the best of FAANGs projects compared to the median of overall developer jobs, which isn't a fair comparison.

I like how the original example is "median" and not the "lowest of lows". While my examples are outliers. HN bias again.

If not algorithms then what would you use to test a generic programmer? You would have to setup a custom panel based on each candidate. A person who has only worked for PHP would need a different panel from someone who has worked on C++ all his life. Algorithms is language and framework agnostic. It doesn't matter if you work in embedded or data science. It doesn't matter if you code in C or Python. The same question can be used to judge everyone.

Companies that don't do the above usually advertise requirements for X technology with Y years of experience. A smart programmer without knowledge of X can learn it easily, but he won't even be called for an interview.

Maybe they can figure out a less anxiety-inducing format for testing algorithmic knowledge instead of whiteboarding?

Radical idea: don't test them. Train them.

If I was interviewing a python programmer, I would also know python. I would ask them about a project they’ve done in detail, see if they are bsing, gauge their ability and experience from that

Your argument is flawed in that it assumes knowing algorithms has any correlation to doing well on the job. It doesn't really matter that it's language agnostic if it doesn't test anything useful.

Programming in general is language and framework agnostic. Someone who can write an extensible and maintainable C++ application can most likely do the same in C#, Java, Rust etc. There's decades of best practices and knowledge around what makes good software and almost none of it is specific to a language.

Most devs use a handful of algorithms in their entire career. And any algorithm they do use was probably adapted from pseudocode on Wikipedia.

It really comes down to what do you want the developer to do? Are the things you're testing them on even relevant to the job?

I'd argue what most companies are testing through their interview process is largely irrelevant.

If it's a generic programmer, then it's perhaps a position where's there's a range of technologies, maybe a range of languages to pick up and a range of existing software to support. Well in that case you want someone who's quick to learn and eager to train themselves, not afraid to ask questions, not afraid to admit they don't understand something, etc. Whether they can write quicksort on a whiteboard is probably the stupidest thing you could interview them on.

I think the best kind of interview isn't overly technical, but rather one that prods the interviewee to think about problems. Describe a fictional system, what would they suggest to make it scaleable? What would they suggest to improve performance? What would they suggest to reduce the time between writing the code and shipping it. Do they understand what a CI/CD system is? Are their first thoughts "rewrite everything!"? Do they think about iterating software development or are they the kind to dump 200 files on someone for code review?

Would you agree that interviewing in psudocode is a good idea then?

But what's the content they should be interviewed on? You say no to quicksort. How about date parsing? That shows you have to handle messy cases.

What do you want them to write if not algorithms? Maybe we have a different expectation of what "algorithm" means? The term is pretty generic, but I feel captures more of the cross language stuff.

Compare a "How does javascript handle equality?" question which falls under "trivia", vs "Implement a hash table", which isn't too hard to do for a naive solution, and demonstrates that you know how the basic data structures you work with day to day work.

>But what's the content they should be interviewed on? You say no to quicksort. How about date parsing? That shows you have to handle messy cases.

But why? The vast majority of programmers don't ever need to write code to parse a date. It's just another test completely irrelevant to the job.

>Compare a "How does javascript handle equality?" question which falls under "trivia", vs "Implement a hash table", which isn't too hard to do for a naive solution, and demonstrates that you know how the basic data structures you work with day to day work.

Both are pretty bad interview questions. Is this a Javascript position? If they don't know how it handles equality do you really think that is the end of the the world for the interviewee, does that make them a bad programmer? It's something you could explain to them during the interview in under 5 minutes (and you should, it eases the pressure on the interviewee). So really what's the worth of asking it?

Likewise, "implement a hash table". Pointless. Ask them "Do you know how a hash table works under the hood?", chances are they might not know exactly how to implement one but they do understand roughly how it works. If they don't know then describe a simple implementation and quiz them on why it's built this way, do they understand the "why" even if they don't understand the "how". Most competent programmers can implement a hash table given an explanation of how it works which is why I don't think it's a good question to ask someone during an interview. It's just a memory test, not a test of how good they are as a programmer, how good they'd fit into a team, how good they would be at designing a large system, how good they'd be at weighing up the pros and cons of different solutions.

This is why interviews should be discussions, not tests.

> vs "Implement a hash table", which isn't too hard to do for a naive solution, and demonstrates that you know how the basic data structures you work with day to day work.

I disagree, anyone can write out an algorithm from memory. That's not hard, and does not prove that you know what you're doing.

I've done my small share of interviewing, and I always did a little bit of whiteboard coding, but the objective was never to see if the candidate remembered algorithms, but to see if they could execute code in their head.

Because I've encountered "programmers" who clearly couldn't execute code in their head, and whose idea of programming was copy-pasting something and heuristically modifying it until it did what it was supposed to do, instead of knowing what the code does. And you really want to make sure you screen for this group so that they fail the interview.

> Describe a fictional system, what would they suggest to make it scaleable? What would they suggest to improve performance? What would they suggest to reduce the time between writing the code and shipping it.

These questions are very senior dev focused, not so sure I'd ask a junior candidate much in this direction. But I always did a question like this when I did my share of interviewing. And the idea is basically to see if the candidate has been around the block. Have they seen the sausage being made, or do they only have academic understanding of the software engineering profession?

And being on the other side, some of the best questions I've gotten was always like "describe the weirdest problem you've encountered". Talking about unexpected shit usually leads to a fun discussion, and you can showcase your ability to problem solve.

why does it have to happen in a high pressure situation with only 45 minutes though? there's no analog to that in professional coding.

HBO's Silicon Valley is dramatized, but I can't be the only one who's coded from the the convention hall floor/hotel, or had to setup servers "live". Every time a site gets unexpectedly popular (/. effect; HN hug of death) and dies, and comes back, there's someone at the other end who's probably close to tears trying to get their site back up ASAP.

That doesn't indict the fact that's the exception and that software programming almost never looks like that.

The real problem is that take home problems are trivially gameable - it's obvious to someone unscrupulous who's not a well-seasoned coder, that it would be well worth paying someone else to do their take-home-interview-homework. (Sure, that would come out eventually, but that's only one of the problems I have with take-home assignments.)

Some places have "open book" interviews, where it's like "here's a laptop, do this thing, and google/stack overflow/devdocs" your way to victory, which is at least better than the older "how much trivia do you remember about this API"-style questions. But I have no idea how to go further and make that unbound by time when an employee is on the other end of the table listening to the interviewee's thought process, if only because it makes it very hard on the interviewer to plan their own time. (Especially considering maker vs manager time.)

Have you ever actually tried a take home exercise? Or is this just based on your perceived concerns?

In the years I have been doing take-home exercises for candidates, I have yet to see a single instance of a great take home leading to a terrible on-site or hire in terms of technical skill. There have definitely been great take-homes for candidates that we later chose not to hire, but usually due to challenges interacting with the product team, or in some cases too much fixation on technical perfection with no focus on business realities, etc. If there are a large number of candidates paying others to do their take homes for them, they should be asking for refunds.

Second, even if this is a problem, there's a very simple solution. For candidates that do well on the take home, either on a call or on the on-site, do a live code review with the candidate and ask them to explain a bit more about why they wrote the code the way that they did. Ask them questions about other approaches, or what they'd do if they have more time. Candidates will be significantly more comfortable talking about their own code they wrote. It'll be quite clear if the candidate didn't write it themselves.

I had a project give me a take home exam with a link to the git repo and basic instructions on implementing a simple algorithm, as well as tests, and FFI bindings. That took a couple hours, then I had two 1-hour remote interviews discussing the code. I thought the process was pretty good: even though I was annoyed at having to do the take-home bit, it served to demonstrate that I could navigate the project source code, build files, learn the FFI interface, etc. The algorithm itself was fairly simple to implement with a little thought, although they probably didn't care if I looked up how to solve the problem on Wikipedia. Personally I think that being able to figure out where the tests were, how to add FFI, etc., was one of their biggest requirements.

Anyway, I thought it was robust but not too time-consuming interview process. I think such an interview could use a reasonably-sized open source project for this, instead of their internal code base.

We have a medium-difficulty coding question which can translate into any language. It's pretty open-ended, in that there are a few different ways to tackle the problem. It's not algorithmic, but it definitely forces the candidate to flex their coding and problem solving skills. From watching how the candidate proceeds, I get a very good idea of their skill level and how they react when they hit a roadblock: are they able to debug? Do they get overly frustrated?

From this relatively simple question, I get a lot of insight into the candidate's skill level, i.e. how well do they write code? and it has nothing to do with algorithms.

For an average shop that pays average salary and doesn't offer anything interesting, the answer is no. For a company that pays way above average,has some decent perks and stock options,why not?

It seems pretty obvious to me that this is an arbitrary filtration process (no different than any other arbitrary filter) that strokes the ego and appears to be better than a random coin toss, but it really is not.

Amazon is the king of assessing how much you want to work for the company. The recruiter will place a huge emphasis on nailing their Leadership Principles, and coming up with stories from your work to demonstrate you embody the Principles. They will even tell you that specific interviewers have specific LPs they will probe on, but in the actual interview they ask very veiled questions and you're supposed to know that they're asking about an LP. You're expected to have a story all laid out in the format they expect; it felt very much like a show of how much you're willing to conform to the interview experience they expect, with no interest in actually getting to know you.

is there a centralized place with information like this but for other companies as well?

Glassdoor? Maybe, not sure if that is exactly what you are looking for..

I mean, yeah. They have a review(s) section for people who used to work there, and a section on interviews too.

Dig up some fake creds on BugMeNot and poke around Glassdoor

Not that I'm aware of, because it's a difficult problem. First off, reports need to be anonymous to prevent employer retaliation, but if they're anonymous, then how can the information be verified? Even if it's verified, without extensive input, it's impossible to say if the details of someone's experience is the norm for the company or just about that one part of the org.

Then finally, happy people generally don't go out an write a huge essay on why things are fine. unhappy people will go on-and-on about every bad thing to anyone who listens.

Thus, we're left with disorganized reports on various sites to glean culture. Eg HN comments on the story about Amazon suing Brian Hall. https://news.ycombinator.com/item?id=23461326

For those looking to move within the job market, it would be worth it to talk to a tech recruiter for a few hours who's job it is, is to know company cultures. (Traditionally, if you, the new employee that they managed to get into a company with a bad fit, leaves before 90-days are up, they don't get paid.)

> it would be worth it to talk to a tech recruiter

IMO this is very good advice. Having been on both sides of the equation, I think a good tech recruiter (emphasis on good) brings a huge value to the process. As a hiring manager, I had one guy I knew I could rely on to bring me really qualified candidates every time. His own filter was excellent, and he only brought me people that were a good fit. I almost didn't need to interview them. Other recruiters were terrible, one even sent me a candidate who had faked their first phone screen so they weren't even remotely qualified for the job.

That is the guy you want to know. He has earned his reputation. And it cuts both ways, too -- his success isn't just when the company is happy with him, but when candidates are too -- all the best ones are his clients, after all.

Blind[0] is a message board with a lot of content focused on FAANG interviewing.

https://www.teamblind.com/

glassdoor has a very good anonymous feedback system for interview experience

This approach also works realy good when interviewing with other companies. I used the STAR [0] principle everytime after Amazon, it almost always worked. Having interviewed a couple of people at Amazon myself definetly helped.

[0] Situation - Task - Action - result, basically your story tellling format for Amazon interviews. Back the day it was even covered on their interview prep page.

STAR is not just for Amazon. It's a great way to answer "behavioral questions" at any company. On the other side of the table, as an interviewer, I can recognize the STAR format instantly and am very appreciative that the candidate has taken the time to structure his or her answer in this easy-to-digest way.

Also, and I don't know if there is a nice acronym for it, but I like the formula where you:

1. Say what you're going to say

2. Say it

3. Say what you said

So if the question is "Tell me about some things you consider when you need to do task ABC," You structure your response like this:

"I'm going to describe the top three things I consider. 1. Foo. [Details here]. 2. Bar. [Details here]. 3. Baz. [Details here]. In summary, Foo, Bar, and Baz are the things I consider in that situation."

Is a much better response than unstructured word salad that contains the same detailed content but jumbles it all together and kind of trails off, leaving the interviewer wondering if the candidate is done talking.

Like that one, too!

Remind me not to interview for that (those) team(s). I am really the type of person that dislike drinking corporate koolaid, or any type of slogans or political line that government or media otherwise pushes. That basically sounds like half way to a cult.

To be honest, it does feel cult-ish from the outside. But as some other commenters have pointed out, they have their reasons, ideally to remove bias from hiring process.

When I interviewed at Amazon last time, I paused all my other work, even my full time job, to focus on it.

Wasn't well treated at interview day (people forgot me in a room for several hours for example, also had interviewers arrive late, among other issues), and probably didn't pass because there was some questions about LP that I couldn't answer, basically all interviewers without exception asked the same question on how I handled an issue in a large team, except I never worked in a large team, whenever I told them that, they would ask the same question, but worded differently.

It became obvious to me that I should have lied, should have written before going to the interview some fictional stories that I could use to answer the LP questions.

Amazon's a fucking pit of vipers anyway, dude, you're better off. If your potential future teammates were so busy putting out fires that they couldn't meet up with you on time, that team was probably in bad shape too

Weird but fascinating. That question suggests:

1. They have a lot of problems with people who come and can't get stuff done in a giant organization

2. They don't try to teach newcomers how to get stuff done in a giant organization. Which would be the obvious answer to those problems.

I'm admittedly stretching a little, but I find more often than not the kinds of questions people ask tell you a lot about what they're trying to select for. (Or, for a lot of orgs, that they have no idea what they're trying to select for.)

I personally know many people who conduct interviews, provide interview feedback, and make hiring decisions at Amazon. None of them operate the way you're describing. There's no "gotcha" as far as making you guess what LP they are trying to get at, or conforming to the Amazon borg. They are simply evaluating you based on your work accomplishments and aptitude, as demonstrated by the projects and experiences you choose to talk about.

Regarding the LP interview format itself, it's just a vanilla Behavioral Interview. See: https://www.thebalancecareers.com/behavioral-job-interviews-.... It's becoming increasingly uncommon in tech (because people are preferring whiteboard coding instead), but it's one of the most common interview formats you'll see in most white collar professions.

A counterpoint: I work with a lot of ex-Amazon employees who continue to conduct interviews exactly in this way. Even my own interview, if I hadn't prepped a story for each of the LPs, I would've bombed.

> Even my own interview, if I hadn't prepped a story for each of the LPs, I would've bombed.

Not denying this part. Candidates at most companies are openly advised about the Behavioral Interview format, and told to come prepared with a collection of stories. Even Google's previous head-of-HR tells all job-seekers to do exactly this (https://www.linkedin.com/pulse/win-every-interview-6-steps-l...). This isn't unique to Amazon in any way, and Amazon might actually be better because they give you very strong hints as to what the questions will be, instead of expecting you to know this implicitly.

My previous comment was mostly responding to stuff like "ask very veiled questions and you're supposed to know that they're asking about an LP" and drinking the Amazon kool-aid.

I'm not sure what I can say other than, maybe I blacked out during my interviews, but I don't remember any interviewer at Amazon asking anything remotely straightforward about any of the LPs.

Maybe it was the group I interviewed with? I do know the position had been open for many months, so maybe they were just terrible interviewers.

> Regarding the LP interview format itself, it's just a vanilla Behavioral Interview.

Not when I interviewed with Amazon. Most companies don't tell you which questions you may be asked about. In my Amazon interview, the questions were directly from the LP. Not disguised as such. Not even rephrased. Example: "Tell me about a time you disconfirmed a belief." This may be a good thing in that it helps the candidate prepare, but it's a bad thing in that often the interviewers are just following a script.

I've never had a behavioral interview that was not a conversation. Only at Amazon was it a "drill". Rapid fire questions, with little to no scope for exploring a topic. It wasn't an interview, it was an interrogation, mostly phrased to put me on the defensive.

> Even Google's previous head-of-HR tells all job-seekers to do exactly this

I agree it's good to come prepared with a bunch of stories. However, the behavioral interview I had at Google was nothing like that at Amazon. It was actually a 2-way conversation.

> Most companies don't tell you which questions you may be asked about. In my Amazon interview, the questions were directly from the LP. Not disguised as such. Not even rephrased

The fact that Amazon told you what questions you'll be asked, whereas other companies don't, most people would say that is more interviewee-friendly. I certainly wouldn't see that as a negative.

Funny enough, the other commenter had the exact opposite experience as you. "I don't remember any interviewer at Amazon asking anything remotely straightforward about any of the LPs". Just goes to show that YMMV.

> Only at Amazon was it a "drill". Rapid fire questions, with little to no scope for exploring a topic. It wasn't an interview, it was an interrogation, mostly phrased to put me on the defensive.

Sounds like you had some bad interviewers. That certainly doesn't excuse Amazon for not better training their interviewers, but what you experienced is a bug not a feature.

This corroborates my experience interviewing at Amazon. Fortunately the recruiter prepped me well on what to expect and the LPs I should study. I was happy to "study for the test", and the interview went well. They even allowed me to bring a "cheat sheet" with notes about the stories I planned to tell, and I made myself a little index mapping stories -> LPs and vice versa. It was great practice and I carried those techniques (keeping the cheat sheet in my head) to other interviews, even though other companies don't put quite as much emphasis on LPs as Amazon does.

The questions didn't seem too veiled, but maybe that's because I knew what to expect.

>They even allowed me to bring a "cheat sheet" with notes about the stories I planned to tell

Is this unusual in your experience? I don't think I've ever been to a job interview where I didn't bring along a notepad where I had prewritten talking points, and also used it to take notes during the interview. I thought this was standard.

I interviewed with Amazon and I hated the LP bullshit. I hated it so much I gave a ton of feedback and had a very lengthy and illuminating conversation about the process with a friend of mine who works there.

Amazon knows full well what the LP interview experience is like, and it's by design. It's not about "how much you want to work at Amazon", but it's also explicitly, intentionally not about "actually getting to know you".

My friend told me that every Amazon employee has to go through an interviewing training course where they are explicitly told they are not allowed to ask any questions that are not directly related to an LP. In most cases, they don't even get to choose the LP that they are asking about. They are given a specific list of 2-3 LPs, and told that when they come out of the interview room, they must have asked questions about those 3 LPs, taken the answers down as notes, and are explicitly told that they must not ask any other questions or take any other notes.

The explanation I was given is that this is all an intentional attempt to make the interview process as impersonal and consistent across candidates as possible. They intentionally do not want to ask any questions that would lead you to talk about your education or hobbies outside of work, and they intentionally do not want the interviewer coming up with their own, not-pre-approved questions. Apparently they think that this makes it less prone to bias, which I suppose is a good thing in theory, but in my opinion, the execution resulted in a piss-poor experience as a candidate.

Apparently it isn't fun for the interviewers, either. My recruiter warned me that my interviewers would likely be rushed, impersonal, and focused entirely on recording notes about our conversation, because apparently if you are designated as an interviewer you have very strict quotas on the amounts of notes you have to take, and then as soon as the interview is over you have to rush immediately to another interview where you again have to take a certain amount of notes. The entire process sounds awful for everyone involved.

It sucks that you (and numerous others in this thread) have had a bad experience.

As an ex-Amazonian that did my fair share of interviewing, I can share additional context on some of the above:

* Tardiness and just the general unprofessional handling of someone being interviewed: At least in the area I worked this would have been unacceptable and the interviewer would have had a very uncomfortable conversation with their leaders. I can't imagine a repeat offender lasting long. There was a huge emphasis placed on the importance of the hiring process, the importance of doing it well, and that finding the right people was about the most important thing to be doing. Obviously at a company of >700k people it's not going to be a consistent experience across the board. * I learnt this from various forums before I attended my own interview so it's not exactly a company secret: but yes, each interviewer is told the LPs they'll be asked to assess. You're expected to spend the full 55-60mins asking questions on those LPs to maximise the change the person gets the job. It's too easy to burn 20mins talking about hobbies and ultimately that's not going to help the candidate get the job. Even worse is that if there's suddenly some shared interest and experience that conversation derails into enjoyable but more unproductive conversation. The feedback skews to "I like this person" and is filled with bias vs "here is the examples for why they'll be successful here". * The standard set of questions is a leveller to give everyone a consistent experience and try to reduce the likelihood that someone gets a job just because they interview well. * Likewise the seemingly unrelenting questioning that's effectively trying to gauge examples for just 1-2 LPs in my experience provides a fairer experience to most other alternatives I've experienced. People with pre-canned answers that they're able to spin nicely start to fumble when people want to keep digging into the details on the same thing for more than 5mins. Conversely people who struggle to think of the right words off the top of their head, or are especially modest about their accomplishments, don't immediately suffer. They're given a time and space, and ideally a lot of guidance, to expand on things. * There's no quota the amount of notes you need to take. However, the debrief for every single interview loop I was part of was always 30-60mins. You don't get to show up with some vague feelings about what the next step is. You need to come with an opinion, with the evidence to support it, and not waste every other interviewers time. If you disagree with another interviewer you need to be able to point at the specific evidence you have either for or against the candidate. We'd often be told "absence of evidence is not evidence of absence". You needed the evidence for whatever point you wanted to make, otherwise you've just wasted the candidates time. * The rigour in the post-interview process and debate, which builds upon the foundation of all of the above activities, is ultimately what it's in service of IMO. Every candidate gets case heard with all of the most compelling evidence that can be provided. I've been involved in debriefs where all but one interviewer came with a "not inclined/do not hire" decision. The discussion and debate that ensued and the recalibration from the collective notes ultimately flipped everyone to a hire. I'm pretty sure every other company I've worked at would have had such a debrief be over and done with in a couple of minutes and we'd move on to the next interviews. * The other part of this that's not always immediately obvious is it's a pretty binary assessment. You can either do the job or you can't. If you can, as determined by the debrief, you get an offer. So first through successfully wins. There's no stack ranking candidates against each other and trying to apply some subjective assessment of who is in a relative sense the "best". Which is again why there's such a strong emphasis on consistency, removing bias,

It's definitely not perfect. It can skew very heavy towards feeling impersonal and like the candidate is being interrogated for an entire day. The thing I missed most about vs other places I've worked was the interrogation feeling like a reinforcement of the power imbalance inherent in a job interview vs levelling the experience and making it just as much for the candidate "would I want to work here/with these people?". But on balance it's probably the most consistent and fairest approach I've experienced (when properly executed, which it wasn't for you, so I'm sorry you saw the worst of it).

I don’t understand what is wrong with hiring someone the interviewer likes. I can see there may be racism, misogyny etc at play here. But setting that aside for a second, if people like each other, all other problems can be solved much more easily.

Because you end up hiring likeable rather than competent people. And you can’t just “set aside all that for a second”. You’ve made a bad hire, at the expense of hiring someone else who could have actually done the job.

also liking is usually "liking people like me" which is code for a lot of bias

Because liking someone doesn’t mean they can do the job. Just because you’re excited about a problem domain and someone else seems to also be excited doesn’t mean the can or will do the job well. They probe for past examples of you doing a job well because it’s a proxy for you doing a future job well. They acknowledge any interview process is imperfect but they also decided randomly pointing at resumes and saying “tell me about X” either doesn’t correlate or correlates negatively. This is in addition to the bias problem which was the other big reason they gave.

I mean, read between the lines. The company doesn't trust that the interviewers' intuition is good enough to beat their formal process.

(And they're probably right)

Well, you’ve also touched on one of the aspects of life at Amazon that isn’t one of the LPs: Mechanisms are better than intentions.

Good intentions fail at some point, even when people have the best of intentions. Being distracted. Trying to do too much. Whatever. Where possible it’s better to design the environment and system so that the right things occur (or are more likely to occur) irrespective of an individuals good intentions.

So yes, you’re right!

I'm pretty sick of Amazon recruiters.

They often initiate contact. I respond. And they don't respond back.

Even worse is when they do. Several times in the past year I had a back and forth with some recruiters, did an online and/or phone screen, and they said "OK, you've passed all the screens, and we're ready to schedule you for an on-site. You'll get a call to schedule it." I never get the call and the recruiter stops responding.

Then the next recruiter that talks to me asks "I see you were ready for an on site but didn't have any. What happened - did you withdraw?"

My response has always been "You tell me. The recruiter stopped responding to me."

This is always followed by profuse apologies. And then it happens again.

A friend of mine suggested I tell them "Can you ask the previous recruiter, and once you find out put it on my account?"

A few days ago I got this message from an Amazon recruiter (his first contact with me). After the initial introductions on who he is, etc:

---------------

Good Afternoon, I hope that this message finds you well and with an opened mind. My name is * and I am a Recruiter with Amazon’s Worldwide Operations – HR team We have an immediate need for a Software Development Engineer (SDE) with Amazon. Your background looks like a really good fit. I'd like to see if the role aligns with your next career move.

We currently have multiple locations; Seattle, Washington and REMOTE, we will provide relocation assistance if applicable.

I have included a link to the full job descriptions to give you more details on the team and position.

If you would like to move forward in the interview process, please provide answers to the intake questions down below and I’ll follow up with you on next steps. If you have an up to date resume please send that as well.

(1) The best phone number AND email to contact you?

(2) Do you have any pending offers at this time and, if yes, what the deadlines are?

(3) Do you need, or will you need in the future, any immigration-related support or sponsorship from Amazon in order to begin or continue employment with Amazon and when does your sponsorship expire?

(4) Are you open to relocation?

(5) What are your salary requirements(range)?

(6)What coding language do you have the most experience with?

(7)Do you have experience using Cloud Platforms(AWS, MS Azure, Google Cloud)?

(8) Have you recently been in contact with someone at Amazon?

(9) Is most of your experience Back-End or Front-end and what Percentage?

(10) Do you have experience with System Design and how many years?

(11) Scale 1-5(5 being the best) how comfortable are you with Data Structures and Algorithms

For some reason if you are happily employed but know of someone with a similar skill set please send them my information.

Thanks

---------------

So let me guess: I'll spend the time responding to all his questions, and never hear from him again. Sorry, not bothering.

I never get the call and the recruiter stops responding.

I expect Amazon has a high turnover of recruiters too.

would you be willing to study for months, go through mock interviews, read books, test your skills etc. to have a shot at working for us

The vast majority of people who work at FAANG type companies do not do any of this.

You'd be surprised. In an industry rife with impostor syndrome, and the perception that it takes intelligence to pass a FANG algorithm interview (it does, but hard work and discipline beats raw intelligence), there's a strong disincentive to admit that "hey, I studied really hard, for months in order to get this job" when there's an expectation that your coworkers and peers will look down on you after this comes to light.

"Grinding leetcode" is a phrase for a reason!

You would think that if the interview process was filtering on anxiety levels it would lead these companies to be full of people with high confidence which would mean imposter syndrome would not be a widespread issue.

Impostor syndrome is most present in elite environments though. It's a syndrome because someone is genuinely talented but surrounded by high talent for each person it's easy to identify an area where they are better than you. In some cases it's easy to feel like you're not the best at anything in your work social circle.

Some people have high anxiety levels but still perform well under pressure. The anxiety gets sublimated into other behaviors, or bottled up until after the interview, or meditated away, etc. Even people that can handle high anxiety levels can still get impostor syndrome.

Filtering on anxiety levels doesn't mean you're selecting for confidence generally, it means you are selecting for people who are not anxious in the interview. If you practice for 1-4 hours per day for a month or more (as I did last time I was on the market) on a particular skill (like tech interviews), you become more confident at that thing. Grinding tech interview practice works, which is not great, for either side, because it's mostly orthogonal to actual job performance.

I think you are very wrong. Every single software person I know at a FAANG spend multiple weeks studying leetcode style questions.

Think about it: you are passing an exam. Why wouldn't you study the questions that you know are going to get asked?

It got even worse lately since every candidate does this, it raises the bar for the expected solution.

Maybe the people you know there are not that great.

Whiteboarding algorithm puzzles with code really does not define the greatness of an engineer. If people are practicing questions to be able to think along those lines and that is proven to work, maybe it really isn’t a test of greatness anyway.

This is the exact attitude that makes people have terrible interview experience.

Looking at your history, looks like you work for Google and bought into the cult that you must be great because you passed the interview.

maybe the people you know there don't want to admit they spent some time studying questions? Maybe they bought into the cult of the "Google only hire the top 1%?" (which is very far from being the case).

FAANGs interview are the most predictable tech interview in the industry, why wouldn't you do it?

Looking at your history, you are biased against Google and bought into that cult. (this works both ways, you're not making a point by attacking me personally).

Looks like you got rejected and found a reason to attack everyone who works there. Like I said, this works both ways.

How could you say that, they work in FAANG!

> It seems pretty obvious to me that this kind of interviews are simply there to assess how much you want to work for a certain company. As in: would you be willing to study for months, go through mock interviews, read books, test your skills etc. to have a shot at working for us

Or in other words it tests how much free time you have to spend on unpaid activity, which can be a good proxy for ageism.

No kids or mortgage? Then you're a perfect candidate to spend countless unpaid hours studying for months, going through mock interviews, reading books, testing your skills etc. to have a shot at working for us.

This works well for tech companies because many don't care about your experience with languages and frameworks of yore. More so whether you can put in 80 hour work weeks.

I've been told to reject a candidate because "he didn't seem that excited to work here"

As someone who loathes any kind of drama and deliberately stuffs down my own emotional reactions as a result, this would screw me every time. My natural response is to avoid getting outwardly excited.

Excitement for the company can be shown outside of emotional reactions, like writing a very company-specific cover letter, or writing a thank you email after the interview.

I have rejected candidates for that. When you're hiring for a startup it's absolutely vital to have people who believe in the mission and the product and not just doing the bare minimum for a paycheck. If that's all you want, go work for the government, or Facebook.

Hard disagree. I’d encourage you to reflect on your startup’s culture and figure out what distinguishes a good engineer from a good engineer who can convince you about their passion.

Usually, it’s when startups want employees to work longer hours than the value the they’re calculating on receiving from the equity+pay that’s being offered.

Facebook has the same vital requirement that you believe in its mission. You wouldn't make it past the screening stage if you didn't care about that.

What exactly is Facebook's "mission"? Would I be screened in if I firmly believed in invading people's privacies?

That depends. Is an ICE vehicle manufacturer's mission to pollute the Earth's air or is it to make safe, comfortable, and affordable transportation?

Safe? Air pollution has been estimated to kill about 200,000 Americans per year.

With the recent VW scandal of modifying software to cheat at pollution tests we can eliminate safe in for that company.

Anyone remember when Ford accountants calculated that a Pinto defect in which people would burn to death would be more expensive to fix than the lawsuits so they decided to not recall the car?

"Profit from selling cars" would be a much more honest mission statement for most of them.

Its mission to sell advertising? People on both sides are lying when this question gets asked and answered.

>it's absolutely vital to have people who believe in the mission and the product and not just doing the bare minimum for a paycheck

Allow me to translate: work indiscriminately long hours in their prime years to make someone else rich from a product or service that will close up shop as soon as the VC money dries up.

So you hire good actors/liars?

I had this one interview some time ago where instead of having the HR person talk about what the company does and prod me to see if I was curious/interested in what it is they do, the opposite was expected to happen. That obviously didn't go well.

After the usual "tell me about yourself" stuff, the HR rep asked me what I knew about the company and was noticeably disappointed when I said I had only accessed the homepage and Glassdoor reviews. She swiftly ended the call after that, leaving me with "homework" to go through (go through more of their web page/web presence, meditate about their mission) after which I would be qualified to resume the screening call.

Later I re-read their Glassdoor reviews and they had other people had had basically the same experience. Strangest interview I ever had.

From your perspective why wouldn’t you spent 15 minutes learning about the companies and it’s products?

I did, they wanted more.

15 minutes gets you a general outline of what they do, a superficial notion of culture etc. I had that, but they wanted more in-depth knowledge of their products and culture.

> but they wanted more in-depth knowledge of their products and culture.

Well, you dodged a narcissistic bullet, that's for sure.

Maybe I'm too old and jaded, but I can't stand corporate bullshit. I don't give a shit about mission statements or your product portfolio. Who has time for that shit?

Yea I remember in college when I was applying for my first jobs I would read through their mission statements so I knew what words to prioritize saying in the interviews. It's all part of the game.

It's both. Either you're brilliant and can implement Kahn/KMP/Rabin Karp from memory or just by thinking about the problem, or you've practiced a ton and dedicated lots of work to the grind.

For large companies like FB or Goog, if they lose a few people to false negatives, it's fine because there are hundreds more lining up.

Either way, from the large company perspective, the system works. Most notably, it's really efficient.

The issue is when small company A starts asking you to implement Tarjan's or something. That's where this interview process seems particularly damaging.

Ironically, according Google's former head of HR, Laszlo Bock, their system of technical interviews with brainteasers not only was not efficient, it didn't perform any better than random chance. He said that they conducted a pretty large study of the correlation between interview performance and job performance and found no correlation whatsoever. They would have done just as well picking names out of a hat.

As a result, he pushed them to move more in the direction of "behavioral interviewing," asking people, for example, to describe difficult problems and how they had worked to solve them. That way you at least learn a little something about how the candidate thinks and about their style of communicating with a potential colleague about technical problems.

If I recall correctly, the only formal process that has been shown to actually predict job performance well is standardized testing of job-relevant skills. Maybe companies should spend more effort on identifying the more important job-relevant skills and work with testing professionals to come up with appropriate tests? Then they could save interviews for finding out which of the top performers seem like they're going to be pleasant to work with.

It was made the clearest to me when I interviewed a colleague who had left to another company and two years later was interviewing to return. I knew he knew everything and had the skills for the job because, well, I know him, I’d worked with him before. He’d done this very job previously. Yet he still performed terribly in the interview and (rightly or wrongly) I had to base my evaluation on my knowledge of him rather than his performance in the interview. Interviews suck so much for evaluating people. If you want a job and good career progression the best advice has to be prioritize making friends with lots of people.

Having multiple respectable people that can vouch for you is a much better performance predictor than a 1-day interview. Same reason that recommendation letters are more important than GRE tests for the ones seeking PhD studies.

It does suck that it’s like this in an industry where you have a lot of non-neurotypical people though. People who are very good and capable but on the spectrum, or shy and reclusive, who really can’t do the networking and conference scene thing, etc. Wish there was a better answer than to tell them they just have to be wired differently.

There are other avenues. I've been excited to hire somebody in the past pretty much on the strength of their detailed technical blogging. Kind of a mess in the interview, but I knew they could do the job coming in. (They were great.)

Let's not forget that the type of work in this industry and hours needed in the name of cost savings hinders many peoples' desire for professional development work outside of work.

I don't know about most people but lately, at the end of the day when I make enough progress to meet expectations, the last thing I want to do is anything related to work anymore, even if it's an investment in myself. At some point my life takes precedent over my career.

If delivery expectations are reasonable (i.e. time requirements), I don't mind networking outside of work. I've worked in environments with reasonable expectations and environments with unreasonable expectations. My observation has been that environments with reasonable expectations are in decline in the name of competitiveness and cost cutting measures, passing costs to employees.

This industry is becoming significantly less enjoyable to work in and growing to a toxic point of hypercompetiveness at every layer. It's not healthy for those working in the industry.

That’s really not a problem. If you think about it, everyone wants to help their network by default. And in startups, you care about outcomes. It doesn’t really matter if you’re shy or gregarious if you can make the crucial thing.

It’s a big company problem, not a startup problem.

> conference scene thing

Who can? Conferences are so expansive!

Hiring primarily based on references almost guarantees bias. White engineers will tend to know more white engineers. Asian engineers will tend to know more Asian engineers.

that is not bias, but rather a lack of diversity

It's still a bias - one that's exacerbated by a lack of diversity

This assumes white engineers only knows other white engineers etc.

This is very much not how the Silicon Valley I've been in for decades works!

Of course, you could have diverse acquaintances and friends, but most people are more inclined to be friends with people who share the same race and gender because they often share the same experiences. This puts a black female at a great disadvantage if her friends are mostly other black females because so few of them work in tech.

I would only refer people I've worked with. And they are as diverse as the industry.

This approach does discriminate against people who are looking for their first job. I think that's both unavoidable and fine.

This gap is covered mostly by internships.

Good networking has always been the best way to find work. Or, come to think of it, opportunity of any kind.

I feel in the tech industry, networking and knowing people only gets you so far - typically just to the whiteboard interview. Then you have to solve the leetcode problems just like everyone else. Especially so in larger companies.

FWIW, all of my friends who somehow managed to get into FAANG or other top tech companies swear that were they to go through the interview again, odds are they would fail. This despite all of them being great engineers.

On the other hand you can get pretty far even if you don't any social network but you can solve textbook problems.

Tip for CS students: Get to know your fellow students and really get to know your professors. Try and leave a good impression.

I'm actually a little scared to leave my current FAANG gig for that exact reason tbh. I'm fairly certain I wouldn't make it back in the door without more leetcode grinding + repeated loops than I'm willing to do at this point in my career.

Plus, stack on top of that, I've seen how the sausage is made. I've done 90+ interviews on the other side. At the end of the day, it comes down to how lucky you got in the loop. I've seen some people written off for some bullshit reasons. Similarly, I'VE declined on people who're likely far, far, far more skilled than I am because of the lowers/raises approach and the narrow slice you're tasked with reviewing in the 50min you're given.

idk if it’s just luck. i’ve had loops in college where i failed them. but then later in my career i had times where i got every company i interviewed at. Excessive preparation was the key difference. I do think luck plays a big part but i don’t think it’s the only factor, or even the biggest. (maybe 2nd biggest)

Luck is likely not far off from preparation in terms of effect though. Luck could be divided up into separate parts like: chance you get an interviewer who asks easy problems, chance you get a hostile interviewer, chance your interviewer is racist/sexist/tabs-ist/hates-your-face, chance your interviewer loves you for no apparent reason, chance your interview is going through a divorce, etc..

I've seen some of the best interviewers I know still fail at various companies. Including ones they were given offers to before. Luck might not be the #1 component but it sure feels like it's not a distant second.

There are some components to the interview you can directly affect, and some you can't.

By grinding leetcode, you raise the odds that you'll get a problem you've seen before, or at least similar enough that you recognize the way to solve it without having to waste time by starting from absolute scratch. I guess that's still luck, but this is one area you can increase the odds in your favor.

You can't change whether you get a hostile interviewer, etc. of course.

i agree but regarding your example about failing companies that you’ve previously had offers from: ive seen this as well. but it was a case of hubris where the person didn’t put in the work to prepare as hard the second time because of their previous results.

the shitty thing about interviewing is you have to prepare hard every time you approach it

While that might be the case for people you've known - I've seen it with people who prepared even more than the last time.

I've also seen the opposite - right? People don't prepare anywhere near as much as the last time and still get offers. I've seen folks get offers at Google and spent maybe a few hours preparing. Luck is just a very wild component.

Yeah, I'm 40+ but takes the time to prepare well for interviews and that has been the key for me. Gotten job offers from all the FAANG's without being very smart. Just go on leetcode and practice for a few months when needed.

This does very unfairly skew towards people like me without kids that can take the time to do this.

Being good at LC requires being smart. Most people won't be able to do "justify text" in 20 minutes without looking up an implementation (and potentially memorizing it).

for me atleast that also came with experience. ie that would have been impossible in my first job search. But repeating the process of searching for jobs a few times over a decade, each time i’m preparing for interviews i’m able to build somewhat on previous times to where problems like that are manageable.

Not really, most questions have answers and discussion threads - during an interview you can see people just writing up solutions flawlessly - they just memorised them. I have interviewed 100+ people.

I don't think regular coding questions, LC tell you anything meaningful in an interview.

I'm more experienced now, so refuse doing it and recruiters and interviewers get all funny about it. But please, please interview. I know I'm pretty privileged compared to others but I won't ever do a LC question anymore and tell that upfront to Google and the likes. I have canceled loops because of the inflexibilty of some companies in that regard.

Sometimes they tried sneaking it in last minute - also tells you much.

Can I ask why you refuse? For me it is more of a formality at this point. I know it won't tell a lot about me, except I cared enough to put in the hours to get this job, but still you have to do what you have to do. I'm pretty senior as well but don't really see why I should not do this when everyone else has to.

In general though the tech industry has a really hard time hiring senior people, perhaps for this reason. I was at Google for 10+ years and did 100+ interviews, HC etc and once you talked to the people in charge of the process they admitted that the process is really weak for hiring senior (L5+ in Google speak) candidates. They in many ways perform worse than new grads in coding interviews which is not really surprising.

So Google hardly hires anyone experience before you get to the director levels (hardly as a percentage of course).

The other well know secret is that if all of us would try to do an interview unprepared we would probably fail.

As someone who refuse myself I simply think it is a waste of my time. It isn't fun, it doesn't teach me anything I find interesting and it isn't a good way of filtering someone.

I would rather be with my girlfriend, play videogames after work hours, watch netflix, go see some friends.

I lost a team member to Amazon recently, he spent 6 months total (two 3 months stints) preparing for the interview. He bombed in the 1st try and passed in the second.

That's 6 months of your life studying after hours to get a job ... I really have better things to do!

He is there now and he says how his work is not as challenging as the one we were doing from a technical PoV and also how the culture is very toxic compared to ours.

Anyways... he is optimizing to retire early so props to him I guess.

I don't do whiteboard coding. When I thought it was meaningful, or might be meaningful, I did it even though I'm uniformly terrible at it. If I was great at it, I might do it even though I no longer think it's meaningful.

But I'm terrible at it, and I don't think it's meaningful, so I don't do it. Admittedly, it costs me some opportunities, but mainly at companies I'm not sure I want to work for, anyway.

At this stage I'm an old fart with a long resume, and more often than not I get hired by people who already know my work. I guess if I was as bad as my whiteboarding performance makes me look then people who know my work wouldn't want to hire me.

So I think this is partially why these questions actually have value. They cannot completely be faked, you need some level of competence even after months of leetcode.

I witnessed someone who couldn't code fizzbuzz end up at our company (OK it could have been an acquihire or something) then get fired, and then immediately hired at Amazon at a pretty good office. I always had a nagging doubt this person paid someone to stand in for the interviews. Thinking back, who actually cross checks this kind of stuff? Unless they know each other personally or take DNA/fingerprints it wouldn't be very hard to pull it off.

Even when you prepare for months, your preparation needs to be related to what you'll actually see in the interview to pay off. Because few companies actually tell people what to expect in the interview, even preparation involves luck.

What is the lowers/raises approach? Google isn’t up to the job (pun was not initially intended, but here we are).

I think it’s the candidate vs team. Is the candidate better or worse than half of the team?

Yep. "does this candidate raise the bar, or lower it". I.e., they have to be better than the average at the company already. As decided by you, from 50 minutes of interaction in a completely artificial, stressful situation.

I don't know what company they're referring to but at Amazon, the criteria for hiring someone is that interviewee would be better than 50% of the people currently doing that role at the company. They should 'raise the bar' in order to be hired.

how do you even gauge that? it sounds like you'd need to keep re-interviewing the current people there to make it relative and fair. which of course isnt going to happen so "raising" the bar doesnt make any sense whatsoever

Doesn't this make it (literally) exponentially harder to hire people?

I think that's part of the point. i.e. the usual line about wanting to avoid false positives (bad engineers putting on an act) even if it means sacrificing many false negatives (good engineers who suck at interviewing, or even just simply wasn't having a good day on the day of the interview).

Ironically the modern technical whiteboarding interview often does require the candidate to effectively act and put on a show,

I knocked together a little simulation.

Suppose programming ability is normally distributed with a mean of 100 and and SD of 15 (values cribbed from IQ). A team starts with 10 people drawn from that population.

More applicants are drawn from that population: if an applicant's ability is above the median of the current team, they are hired, replacing a randomly-selected member of the team; otherwise, we pass.

At the first timestep, half the applicants get hired right away. It takes a few hundred total failed interviews to make the 25th hire, and a few hundred thousand(!) for the 100th hire. These have very long tails too: the longest run (of 1000) took 800M total interviews to hire 100 people.

There's no way this rule is actually being applied in practice.

raising the bar would also mean that their pool of people who are merely good enough gets very small very fast and Amazon just doesn't have the reputation to attract the absolute best

Lowers/raises the bar. As part of interview feedback an interviewer is supposed to rate the interviewee as raises/meets/lowers.

Typically, just one lowers in an a loop of 6 interviews is sufficient to reject. On the other hand, a loop with all "meets" isn't a sufficient either. It needs about 3-4 "raises" to overcome just one "lowers". With two "lowers" the candidate won't even make it to the debrief or worse the loop will be cut-short.

The bar-raiser (it's their job) are expected to specifically look for candidates who raise the bar. Raising the bar == better than 50% of "employees" (not job seekers) at their level. The rationale, when the bar-raiser program was created, was that the collective competency in a corporation should increase as more employees are inducted.

Source: I was a bar-raiser at Amazon; 400+ interviews, 300+ as bar-raiser, conducted hiring boot camp, bar-raiser trainings, trained bar-raisers.

How does the organization gauge its success in bar-raising? Did you guys just progressively hire less people over time?

Companies tend to keep track of the onsite success to failure ratio and then if it's too low examine if the bar raising is too aggressive.

Which ends up going in the direction of meetings like "Should we standardize on what questions we are asking in interviews?" or "Should we standardize what pass means or fail means for a particular problem?"

Bar-raiser program didn't have a feedback loop to track its effectiveness. Bar raisers periodically met to discuss topics related to hiring and increasing interview bandwidth but we didn't evaluate the process as such.

That said, the program (like any such program I guess) didn't scale well with the immense pressure on increasing the head-count which in itself was a result of delivering more. Hiring managers and recruiters (who are incentivised to hire more) would rig up interview loops in a way to get their candidates through.

The program itself came under increasing scrutiny (I guess for the right reasons) as being too restrictive. I distinctly remember period between 2008-2012 they hired at a blistering pace, all over the world. In fact in Seattle when they moved into SLU campus the buildings went from empty to nearly 100% occupancy in a matter of year or so. You could even see it based on the number of product launches from 2012 onwards (AWS, the hardware products, new country launches etc.,).

And now Amazon is such a behemoth that it doesn't even make sense to speak about it or analyse it as a single entity anymore. My prediction is in 5-8 years it'll be broken up into at least three companies under Amazon as a holding corporation --- cloud, retail, and consumer devices.

My judgment of Amazon is tainted badly by a bad apple we couldn't fire soon enough - I had the unsavory role to collect evidence for failing the PIP that would inevitably happen. Week after person was let go, he was at Amazon in a pretty good division.

I keep wondering how they hired this person - never in a million years would they have passed a regular hiring chain even with fizzbuzz type questions. Wonder if they had someone stand in for the interviews.

It's often been mentioned that interviewing skills is different from actual work skills. Could be that he's an interviewing wiz but a poor day to day engineer?

Netflix is well known for firing people that don't pull their weight with far more ease than most companies. But a friend who works there tells me there's still people there that perform poorly but somehow managed to get both hired and stay on.

Not unusual at all. Interviews are team specific and people cheat them all the time if they know people on the team. Bar raiser system doesn't work.

> Bar-raiser program didn't have a feedback loop to track its effectiveness

Which itself is a Bar-lowerer.

What about diversity candidates, is the "bar" lower for them? Genuinely curious.

The general approach to dealing with low diversity in a team seems to be to increase the percentage of phone screens being allotted to diversity candidates (which on the other hand reduces the number of non-diversity applicants that get a phone screen).

And not actually lowering the bar during an interview / hiring committee meeting.

If you select for something irrelevant lika race you ofcourse automatically lower the bar on average (and raise the bar for the discriminated race)

While I was there, increasing diversity wasn't a priority for them. Now that I think about it, I don't remember diversity even as a discussion topic let alone efforts to address it.

I'm in a similar position - a tech lead role at a Big N - and have recently been doing some interviews for senior roles at FAANG. The most frustrating thing is knowing I can't really leverage anything I've learned over the past 7 years of my career in the interview, at least at the early stages. I've been "studying" Leetcode after work for a month or so but always seem to make stupid mistakes under the time pressure of the interview. I don't want to spend my entire career at one company, but it's clear I'll need to be very lucky and/or practice much harder on my own time if I want anything like a similar position/salary elsewhere.

This is something I find so disturbing, and I experience similar.

I feel extremely little of what I'm doing in my real, actual, job, helps me in advancing in my career. Unless maybe if I choose to stay at my current company until retirement lol.

Otherwise, why bother doing anything more than the bare minimum to get by at work? It would be a far better investment of time and effort to grind leetcode and practice for interviews, instead of going above and beyond to excel at my job. At least until I get into an "endgame company" where I feel it's worth staying long term.

If you want a good career, working on your interviewing skills is the most important thing. Networking and resume fluff are close seconds. Job skill is almost irrelevant. It's an ugly truth.

sad but true. hope as an industry we can find a better way.

I think the answer is we need more innovation. We need a million small startups instead of a dozen tech giants. Software scales but starting up is a moonshot.

My personal career epiphany was moving from a tech giant to a late-stage, pre-IPO startup. The amount of freedom and range of challenges I dealt with were exhilarating. Did it pay as much? Not in the beginning, but I was given the room to grow.

What is wrong with the tech industry? What are we doing all of this for?

I've been asking myself the same question as of late, I haven't come up with a good answer yet.

supply vs. demand.

as long as 9 other candidates are willing to run the LC gauntlet, the 10th doesn't stand out.

At some rare companies people ask questions about real-world scaling which has been nice to leverage some actual knowledge but that's usually just a small part of the process and almost treated as a soft-skill which is kind of hilarious.

We just really want to hire a senior dev who can spit out highly optimized code that in the real-world would be handled by a stable open source library rather than people who have built stable systems.

My dream is to have some kind of "working for myself" career like the creators of the Apollo or Overcast apps. Not contracting with companies as clients, just working on one small great project for regular people to use and selling it yourself. It sounds fantastic.

Yeah, sometimes you may not be thinking straight, a have a little sleep hangover/anxiety Induced brain fog on the day of, and you can say bye bye to the job.

I would think that this is partially by design to get employees to second guess ever wanting to leave and explore other options.

Stop lumping Apple into FAANG in this way. Facebook, Amazon, Netflix and Google all have centralized hiring. Apple does not and no two groups interview the same and there is tons of latitude for managers to make this lower toil. There is no cross-organization level setting or job descriptions for any position at Apple outside of an SVP business unit. I know from experience working at two other FAANGs that Apple does things way differently. Apple's biggest weaknesses and strengths come from their lack of centrality, but here it is to the benefit of the interviewee since they are given more individual care out at the periphery.

My understanding is Netflix does not have centralized hiring either?

Amazon does not really have centralized hiring. Each team hires for itself, though there are bar raisers from outside the team designed to be a counterweight to desperate hiring managers.

I was this way too. I got into Amazon via acquisition so I never actually passed a real interview process (though there was a limited one as part of the acquisition).

I transferred teams within Amazon and then left for my current company. I do wonder if I'll ever be willing to put in the work to get a FAANG job again but at the moment I really don't care. I like where I am so I'll just cross that bridge when I get to it. If I want it badly enough at some point then I'll go through the steps. If I don't, I'll just find another job.

I don't understand this obsession with leetcode style questions specifically. I just got a job at a FAANG on an algorithm heavy team and all they asked me was linked lists, some basic graph questions, and some basic (and I mean really basic, high school non-calc level) vector math. I think just having a rock solid understanding of the basics will help much more than trying to grind questions and memorize patterns.

> I think just having a rock solid understanding of the basics will help much more than trying to grind questions and memorize patterns.

It isn’t about memorizing patterns, but about hopefully spotting the exact question.

A friend is currently interviewing for a FAANG and the recruiter flat out told him that his best chance of passing was to do a pile of previously seen questions for that FAANG and hopefully he gets it again in the interview.

I think it's both.

If all the celestial bodies align and your luck is through the roof, you'll get exact problems you've solved before with the optimal solutions ingrained in your mind.

But second best is you'll get problems that are at least similar and you are familiar with the patterns, tricks, and gimmicks to go about tackling it, instead of having to try to think up a solution completely from scratch.

Horrible luck? It's a question you've seen and practiced but your brain freezes and you can't remember how you solved it. Or something you've seen and earmarked to practice, but couldn't get to it before the interview day. This has happened to me.

> It's a question you've seen and practiced but your brain freezes and you can't remember how you solved it. Or something you've seen and earmarked to practice, but couldn't get to it before the interview day. This has happened to me.

Same. I had an interview the day before I was scheduled to review binary trees. What was it on? Binary trees.

In 5th grade, I was in a poor school district. My parents had been pushing (with other parents) for a 'gifted program' for years. Finally, in November, a program was being introduced. I was given a test - IQ test perhaps, but an entrance test of some sort. I went home, dejected at the stuff I didn't know - was asking parents what the questions were - "what is an octave?" was one I remember not knowing at the time. I'd "got in" to the program, but... we suddenly moved over Christmas to a newer school district. It was richer by comparison, although still in the same county.

First week at the new school, I was taken aside and given a test for their gifted program. Exact same test, which ... I completely aced, because I'd figured out all the questions I hadn't known in November (from parents). It felt like cheating, and I told the teacher about it later. I don't know why they couldn't have just taken the test results from the previous school - same county, shouldn't have been an issue.

Anyway, if you keep studying known questions, it will increase your chances of passing in the future ;)

That's so disheartening. When I'm looking to work with somebody, I look for somebody that can solve problems (and admit when they don't immediately know the solution, which is extremely important imo) not just regurgitate a solution to a problem they know already. If this is actually what the recruiter said, this is a huge indictment of that company's hiring practices.

This practice seems common. I've had recruiters at several top tech companies (including two of the FAANGs) tell me this during initial chats.

I imagine recruiters actively want you to succeed and get hired because that benefits them too, so they do what they can to aid you in your chances. They'll tell you the official, pre-canned lines about "we're interested in what your thought processes are and how you think", but then they'll also tell you the truth and recommend you take some time to grind leetcode.

By comparison, in a recent FAANG interview for general dev, I was asked to reconstruct a BST from preorder and postorder traversals (Leetcode 889), and also to design a thread pool that is aware of dependencies among tasks. It sounds like the experience really varies!

I had a similar experience. I just accepted an offer for a senior engineering position at a FAANG, and only got a few basic coding questions. Out of eight hours of interviewing, I think they spent six hours on behavioral questions.

It fundamentally depends on the FAANG. Which FAANG was this, or are you just minimizing the difficulty of the problems?

My understanding and experience is that one of the A's and the N stand apart from the other letters in having a less of a "one size fits all" approach.

Agreed. Connections at best let me skip the phone screen and do the onsite leetcode at FAANG.

Maybe I just don't have friends at the >= VP level?

Would VP level help?

Not a close friend, but do have a close associate who was VP level at Google. We were having coffee and he offered to refer me, but said I'd still have to go through the whole standard interview loop - even the phone screen in fact.

Depends on the company. I think the FAANGs basically all require jumping through the same hoops. Optimistically it prevents nepotism, but I think that's less of a worry in tech circles, and the times I've seen interviewers overridden because of connection between tech people, it's generally worked out. Of course, for non-tech (product, marketing and the like), it's been hilariously bad.

I was recently interviewed by a Google bet with a referral from a well-liked VP level employee for a job I was well qualified for (IMO) and didn't make it past the phone interview. /shrug I just blame it on Covid.

It's very different at smaller companies where the whole interview might be waived (or just no leetcode) if you worked with one of the technical leaders or founders before.

One of my ex-coworkers/now-friend is what I call a master networker. His network casts wide across the tech departments of a huge number of small hedge funds in NYC. If he wants a job, he can just call up any of his friends/ex-colleagues at these places and odds are several of them will simply just give him a job at their firm. If an opening doesn't exist, sometimes they'll literally create an opening for him.

The downside is that he has to be very un-picky on what the role entails whether it be working with some super unsexy legacy technology (Excel VBA! Legacy ASP.NET webforms!) or doing devops or even helpdesk duty too. Hedge funds being hedge funds, small as they are, they all pay pretty well, though not at the level of a top tech company.

It's a combination of these places being pretty small shops and having friends there in high places. Including, from what I hear, a billionaire hedge fund exec or two.

Eh, I got my job at a very early stage startup through a friend I met on Twitter. The "interview" was a chat with the (reasonably technical) CEO, where we talked about political philosophy for 45 minutes, then I said something about "oh, I should probably tell you about my experience and resume," so we went over that briefly. No coding challenge or anything. I assume my case is an extreme outlier, though.

But yeah. "Networking" by meeting groups of strangers for drinks probably wont get you a job. But making friends who happen to work in (entrepreneurial) tech might.

It s not, I ve been interviewed that way and interviewed others that way. Many times it led to horrible mistakes so now I give a simple project to do at home with no deadline.

If a guy can do well that kind of well rounded project, even by cheating, I want him and his network all the same.

Startups and smaller companies are a different story. One of my friends got a job at a startup by virtue of standing in line at a cafe holding a programming book, and the (non-technical) founder standing behind him hired him after a quick chat.

Bigger companies often don't let you through like that though. My friends are constantly offering to refer me to roles at their companies, but every one of them would still involve going through the whiteboard interview ritual. For now, I'm not taking them up on the offers because I'm not confident I'll pass the leetcode rounds.

I was similarly once hired in the back of a shared cab.

Another time, apparently saying the phrase "binary tree" was enough to clear the whiteboard and move along to the next question with a gold star.

Another time I was hired after doing zero programming exercises and just telling the company that I had a competing offer (which I did).

I've had some baffling rejections as well.

Am I crazy or are you replying to a comment that is a clear example of how knowing someone can lead getting jobs in spite of whiteboard interviews?

EDIT: Ahh, didn't see the FAANG qualifier.

Not sure I quite understand.

I did use "FAANG and other top tech companies" as an example, but I'm seeing this practice expand out towards many other companies both in and outside the tech industry.

I myself work at a non-tech company (bank) and even with my team/department, all a referral will do is just get you to the leetcode interview and maybe a slight advantage if the final decision is between the referred candidate vs. another non-referred candidate with all other factors being equal.

Tech seems to be the only field that does these dumb types of interviews. Accounting, law, medicine etc seems to work well without these filters. Why are tech interviews so stupid?

So you're advocating a license to practice software development? Because the three fields you mention are all licensed.

Sure, why not.

You do not have to be a CPA to be an accountant. a CPA is an accountant who can sign off on financial statements for an auditing firm.

The whiteboard standard already acts like a de facto license.

Algorithmic/data structure more theory-oriented problems are unrepresentative of "actual work" and are annoying, but what's more annoying is how the process is 1) opaque, and 2) repetitive. So you end up with candidates grinding through 80+ Leetcode problems to both master the skills necessary to ace these problems, as well as covering as many commonly-asked problems in a bid to game the system by preempting their interviewees questions. Goodhart's law abounds. This basically makes the process into an interview version of standardized testing. But- despite the standard prep process and a standard bank of questions, it can all easily fall apart because of subjective interviewer opinion, and so all the preparation is for naught. Candidates end up having to do the interview gauntlet for each company they apply at.

So standardize it. Put out clear, industry-standard rubrics for how one's performance is graded. Don't say "we only want to know how you think" and then ding candidates for not getting the right solution. You obviously care about the right answer as much as the thought process- don't be disingenuous about it. Maybe even standardize the level of difficulty of problems. And most of all:

Make this process a one-time thing.

Or rather, a once every five year thing. Outsource the ds/a interviewing section to a third party testing organization, like what Triplebyte is attempting to do, and have the actual interview be personalized to the company. Ask domain specific questions, relevant experience questions, system design, and culture fit questions during the company interview. And leave the DS/A portion be akin to the licensure tests that other engineering disciplines and STEM professions already have.

Software engineers may hate credentialism, but if it's a credential that you only need to take once or twice in a decade, then it's already far superior to the current system that forces candidates to undergo this each time they change jobs.

There's a difference between licensing and certification. A license is when the government says you need to meet some requirements to practice. A certification is when some entity, usually a board or industry group certifies that you are good enough at your job. We don't need licensing, but it would be great if we had broadly accepted certifications for software development. Even if you believe in FANG style interviews, ideally you should be able to pass one time and have it be accepted by every company. Once you have the certification, we could do interviews that focus on the type of things that individual teams and companies care about.

Not a license to practice, but how about some sort of exam I can pass that would let me never have to do the whiteboard dance ever again?

As it is, I'm basically studying for an unknown and nowhere near standard "certification", more or less, in the form of the technical interview, with only an inkling of what I might have to study for every single interview I go on.

I guarantee my career has been stunted because I delay my job search because I just don't want to go through that bullshit all over again, and stick with jobs long after I've stopped being passionate about them, despite there being a huge demand for software engineers.

Not everyone has to take the standardized exam, but if you do, you can show that to someone and they go "okay, I'm going to skip the technical pop quiz, let's move on to other things".

That's supposed to be what my degree was for, but apparently no one trusts that anymore, so why did I waste the time and money in the first place?

The other fields have various ways to filter out people.

* Accounting has CPA exams/certifications

* Law has the bar exams/law school

* Medicine has the USMLE/medical school

Tech prides itself on the fact that "anyone" can be a great tech worker regardless of how you get there - read some books, attend a bootcamp, go to college, or even just start banging out code on the keyboard until you gain experience

Unfortunately, that means that the companies have to have some filter

"Complete a university computer science degree"

Did that. Doesn't seem to matter for shit in any interview I've had, except maybe to pass an initial HR filter. Waste of fucking money that was (I'm kind of joking, I did learn some valuable things in college).

Parent poster is pointing out that _in addition_ to your degree you must pass a professional exam as well. This is common in professional fields where there is a licensed title (e.g. M.D. or P.E.).

Btw there is a Software Engineering PE exam but I don't know anyone who has taken it: https://ncees.org/wp-content/uploads/2015/07/SWE-Apr-2013.pd...

If you take and pass it you can add the title PE to your name in the US.

I know people with a computer science degree that couldn't write a function of production code if their life depended on it, and I know people without one that are gods on a keyboard.

I think a better alternative is some type of bar-like or CA-like test that anyone can take regardless of qualification, but which you must pass in order to be considered a competent software engineer.

Well, the same is true for lawyers. I know attorneys that have to look up everything in books. And I know attorneys that can cite basically any supreme court verdict from the last 10 years out of their head.

They instead use credentials. Where did you go to school, what was your gpa, where was your residency, what law firm did you work for, how likable are you.

Yes tech interviews could be much better but other industries hiring process is just as if not more broken than ours.

I had a guy ask me the number of hardware interrupts there were on a processor I used 4 years prior and what the exact part number was on an interview. Just incredibly stupid questions.

> I feel in the tech industry, networking and knowing people only gets you so far

I think it depends how influential your connection is.

At my previous FAANG job, my director was tapping his network to back-fill some positions and i heard through the grape vine that she waved parts of the interview in order expedite the process.

it could've been a rumor, but then again the person who joined was terrible to work with so i'm biased.

If you know the right people, they might tell you pretty much what Leetcode questions will be asked, who you might be interviewing with, what their personalities are like, etc. though. That is alot easier than going in blind (for most people I would assume).

The trick is knowing someone who works with your interviewers. If they talk you up to the interviewers ahead of time, it will bias them to be more forgiving, offer more help because "I'm sure it's just nerves", etc.

It's generally not even on purpose, just subconscience bias.

>FWIW, all of my friends who somehow managed to get into FAANG or other top tech companies swear that were they to go through the interview again, odds are they would fail. This despite all of them being great engineers.

Im fairly confident that's at least part of the dominance of this approach in big tech. If the tech companies can't collude with no-hire agreements (as they have in the past) to prevent developers from jumping ship to seek raises (i.e. introducing direct anticompetitive barriers) they can collude on an unspoken agreement to raising the barrier to entry, making the labor market less mobile and therefore, artificially decreasing competition. It would be pretty difficult at this point to make a case that the current hiring trends are intentionally colluded anticompetitive labor market practices but the end result is the same.

I say part of the reason because the mess that is the current hiring process is really a multifaceted win approach for big tech and loss approach for most of the labor force.

I suspect if you had a large movement of mid-senior level developers that wanted to shift positions at the same time and began refusing businesses participating in this practice, the industry might be forced to make reasonable changes.

The issue is that our industry has no labor organization and the probability of a critical mass of labor independently making such a fundamental shift (especially in the current economic client) is near zero.

Interesting theory, but does the game theory hold up? If high interviewing standards are only good for scaring employees into staying put, couldn't one of the FAANGs "defect" and lower the bar, gaining a hiring advantage over the others?

Do they care to have a hiring advantage, though?

Probably not, at least in the sense being talked about.

After all, there is a glut of applicants to the top companies, combined with the idea of wanting to avoid false positives (bad engineer putting on an act) at any cost, even if it means losing out on many false negatives (good engineers who interview poorly).

Not sure if that is FAANG or US but nah; I recommend people and people recommend me to large corps and the interview is pleasant banter and nothing else. If I see a whiteboard I would walk out anyway; what is this, grade school? If my resume and references do not help then I have no interest working for you.

I'm sure there are still companies like that out there, but getting fewer and fewer. My current company (bank, non-tech) hired like that when I joined 5+ years ago. At most you would have gotten some language or framework trivia questions. Fast forward to today, and interviews are now all leetcode on whiteboard - just like a FAANG, albeit easier with lower standards. My direct manager and half my team is in London, so I know any UK candidates also get leetcoded the same way US candidates are.

Same thing with other banks and hedge funds - 5 years ago I think the only major bank that leetcoded me was Goldman, and even that was leetcode easy level. Today, every bank leetcodes candidates AFAIK.

Pretty much every company, both tech and non-tech, I've interviewed with the past 3 years has done leetcode interviews at various levels of difficulty and rigor. Interestingly enough, recently FAANG and a few other top tech companies seem be creating frontend specific interview tracks that are arguably more practical tests of JavaScript knowledge rather than pure leetcode problems.

Maybe time to look for my pension then! But no, I do not have the same experience; when I recommend someone (always seniors though; you sound like one), they do not have to do that. Another way to avoid (in recent experience) is to apply for positions as an agency (so a consultancy company which might be a one person company but looks like a pro outfit, website wise), work for 3 months (none of our people ever got interviewed: they trust us to have done that) and then apply for a job. Skip all the hiring bollocks as you already worked there and with the team etc. Again, not speaking for FAANGs.

Networking can in fact overturn results of the interview loop. In some cases, the result is known and the interview is arranged to support the case. You may outright fail some questions, and still get a green light from those interviewers.

Your advice goes directly against your example and I would say that most companies interview like that. Surely the advice should be to spend time on interview prep regardless of how good you are at the job.

> If you want a job and good career progression the best advice has to be prioritize making friends with lots of people

Getting good at interviewing should also be a priority, and is doable.

Getting good at interviewing is largely about developing (a) genuine programming skill; and b) at least some people/communication skill.

When you have those skills, you have less anxiety about the interview, and can perform well.

Genuine programming skill has no correlation to the skill of regurgitating algorithms in a reality-show type setting. Actual development, of real products, isn't done like that. This is the reason why it's so perpetually controversial to put developers through this type of interviews which measure nothing relevant to the job.

> regurgitating algorithms in a reality-show type setting

You're not wrong, but this falls under the genus of "companies that suck at interviewing," and thereby hurt themselves (i.e. the company). In other words, it's not an inherent fault of the programming interview.

But there is more to it than meets the eye.

Some companies actually just are that incompetent.

But, to take a particularly successful and egregious offender: is Google actually just that incompetent? So that they systematically do hiring in a poor way?

No, of course not.

They actually don't want the best people.

They want the mediocre people who are also obedient. The kind of people who can be told to study an algorithms book, and will do it, and will regurgitate it, as a show of obedience, mostly, but also mediocre (but not great) intelligence---not great, because if they had great intelligence, they wouldn't waste their time on that kind of activity.

Such people can be treated as interchangeable cogs, which is why BigCo wants to filter for them.

> They want the mediocre people who are also obedient. The kind of people who can be > told to study an algorithms book, and will do it, and will regurgitate it, as a > show of obedience, mostly, but also mediocre (but not great) intelligence--- > not great, because if they had great intelligence, they wouldn't waste their > time on that kind of activity.

Perhaps a bit cynical but there is truth to that. A huge company just needs cogs that can do the job and don't think outside the box much.

I interviewed someone that several of my coworkers had worked with before, and spoke very highly of. He even was my current manager's manager at their last job together.

He kinda blew the interview. He did okay, but not even close to the level of experience he had.

It was very obvious he was nervous, so we hired him anyway. No regrets.

At my company we interviewed two people. Both did fairly poorly in different ways. One we hired because he was a referral and came highly recommended despite a not great interview. He's been great.

The second guy was rejected but later hired as a contractor and he's worked out well as well and now we're looking at offering him a full-time position.

My conclusion is that our interview process isn't good at determining if a candidate would be a good employee.

My preference would be to hire someone as a contractor and give them a project then use that to determine their quality and fit. Unfortunately, HR tells me that's not feasible for reasons I don't understand.

> HR tells me that's not feasible for reasons I don't understand

You are asking them to do some work.

My preference would be to hire someone as a contractor

When I was contracting it was common to be offered a job at the end of the contract. I always turned it down because I liked being a contractor. From talking to my peers at the time this was common.

If someone didn't want to be a contractor then they wouldn't accept your offer in the first place.

Interesting. I usually do pretty well in interviews and I've always attributed that to my four years on the debate team in high school. This tells me it might be even more true than I previously thought. Even when I'm nervous as hell, when you put me in front of a judge (interviewer), I click into "performance mode" where I project confidence even if I know what I'm saying is BS, let alone just a bit unsure. This is how you win parliamentary debates when you've had five minutes to prepare for a topic you know little to nothing about—and I guess it helps in interviews too!

That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense (useful when you have 2 minutes to prepare a 3-5 minute speech on a topic you've never seen before). So when I get up in front of a whitebard, I just start talking confidently, and even if by the end haven't solved the problem quickly or efficiently or at all, it sounds like I've been thinking sensibly about it.

My takeaway, I guess—parents, make your kids do debate!

> That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense

You're effectively advocating for more bullshit. The problem is that this works in the first place - we shouldn't be rewarding it.

We can't expect everyone utterance to be rational and logical. We're all just human at the end of the day. We should expect job candidates to try their best to get the job.

Yes we can! We’re computer programmers. Everything can and must be quantified.

What's the point of asking these sorts of questions? Typically to try to get a sense of how somebody thinks and what sort of rapport they have with the interviewer while problem solving, not that they get to the "correct" answer.

I think they have some utility because this is a common occurrence when dealing with clients.

He isn’t advocating it. He is just saying that you must play the system as it is.

I don't know if he is really advocating it, but I absolutely support the notion that programmers should learn to bullshit better.

I'm not sure he's advocating for it, just pointing out that it's true.

> My takeaway, I guess—parents, make your kids do debate!

He's encouraging parents to make their kids learn this skillset. He is absolutely advocating for it.

He's saying "this makes you more likely to win at the game, perhaps you want to teach your kids this skill so they can win." He is not saying "the game is fair."

That is absolutely not the same thing. You have to play the game by the rules, that user is just telling you how to win.

don't hate the player, hate the game. easier to tweak personal actions to address the world as it is than to rant against how the world doesn't work per your ideals. at least this guy is self aware enough to admit there is some amount of bullshit in what he says. I am more concerned with people who refuse to admit this, or worse, genuinely don't believe it.

You're right, unfortunately that's what it sounds like I'm saying. It's not really what I meant. The idea I was trying to get across is closer to "being able to communicate what I'm thinking about as I'm thinking about it, and verbalizing that as a coherent sentence rather than just fragments." Ironically, I wish I had communicated that better :)

I’m a person who finds out what I’m thinking as I speak. Iterating towards an idea with confidence in each turn. Even if you end up in the vicinity of where you started, problem not actually solved, you demonstrate how you move through problem space.

Similar to another comment I read comparing interviews to athletic tryouts and competition, which tries to assess how a body moves through physical space.

Rhetoric was the most respected skill in Ancient Athens.[0] I think we would do well to stop thinking of rhetoric, debating, and public speaking as hobbies for some high school students. They are essential skills for wealth creation and advancement in society.

[0] https://liberalarts.online/trivium-and-quadrivium/

or Model U.N.

> That and debate gave me the ability to just start a sentence without knowing where it will end ... it sounds like I've been thinking sensibly about it.

I've always structured my interviews to weed that kind of thing out.

I'd be interested to hear more about how you do that, and what you try to select for instead.

Comprehension. Ironically, I just started writing a blog post about it 10 minutes ago.

To summarize, I try and get the candidate to discuss tradeoffs between different approaches to common situations that the candidate should be familiar with.

For example:

Should GetUserById(int userId) return a null, or throw an exception, when there is no user with the ID? Explain the tradeoffs; or explain why each approach is better in different situations.

Should you deserialize (xml or JSON) to a dictionary or a strongly-typed object? What if you need to manipulate the document while preserving fields that you didn't know about at compile time?

Assuming you have the experience I'm screening for, you either know the tradeoffs or you don't.

Any open ended question like

> Explain the tradeoffs

will benefit greatly from being able to synthesize a bunch of information out of some simple base. Yea you need to know the technical details but being able to explain tradeoffs is more abstract and the OPs method will just lead to a better answer here.

Exactly, yes! It's about explaining what you're thinking about as you think about it. Silence is a killer in interviews in my experience (not always—you can be strategic—but for the most part), and I try to minimize dead air in any interview I'm in.

Why is it a bad thing? It's off the cuff, and likely less filtered than a prepared statement. If they end up somewhere sensible, then your interview process actually worked; you found someone who when confronted with a new problem, is able to think sensibly about it, even in an interview situation.

Because if you're talking without knowing what you're going to say, you're not thinking about the problem at hand.

The right thing to do is to narrate what you're thinking about. It's very different than starting a sentence without knowing what you're going to say.

"you're not thinking about the problem at hand."

But OP said he ended up somewhere sensible. He -was- thinking about the problem at hand. As he spoke. He got there in a roundabout fashion, perhaps, and had to think quickly, but it's still -valid-. He's not just bullshitting the interviewer (which is very noticeable and easy to suss out, and never ends anywhere sensible).

Duplicating a comment I made above, the idea I wanted to communicate is closer to "being able to communicate what I'm thinking about as I'm thinking about it" — NOT just rambling my way to a BS answer.

> That and debate gave me the ability to just start a sentence without knowing where it will end and then talk my way to a conclusion that sounds like it makes sense

That sounds like a good skill to have in marketing, sales or politics. It sounds like a bad skill to have for engineering.

And an interview is very much marketing and sales of your skills :)

> project confidence even if I know what I'm saying is BS

One of my very worst bosses had this quality. Perversely, it was about the only thing I admired about him.

The guy could walk into a meeting of NASA flight engineers and engage in conversation without uttering a single fact that would betray his total lack of any knowledge/qualifications.

n.b. He was fired not long after I left the company.

To be clear: I don't advocate for doing that, and I don't try to BS my interviewers. That's why I said "let alone if I'm just a bit unsure"—if, during high school debate, I can confidently BS, then now, during an interview, I can confidently talk even if I'm nervous and not fully sure of myself.

An upvote is warranted for a life long gift to kids to be able to communicate confidently.

A downvote is warranted for giving a bad advice on gaming a system and giving both parties in an interview process a poor indication of what is expected in possibly years to come.

Thus, no vote given.

I think given what I read about debate teams -- on podcasts and here in HN, I am on the fence on suggesting that to my kids. There are certainly great benefits, but not too certain if those out weights the drawbacks.

I'm not from a country that really does "debate teams", but as I understand them a lot seems to be about "winning the argument", which doesn't strike me as especially great.

On the other hand, it probably makes people better at spotting a bad/fallacious arguments, and gain appreciation that complex topics tend to have multiple reasonable opinions on it – two things we desperately need more of.

True, "winning the argument" is one of the main drawbacks I fear. The main benefits for me is to foster (often quick) logical thinking and reasoning and the ability to articulate it. It is difficult to build this skill without a "debate team" setting. More so if your kids are more on the shyness scale.

I completely agree with all of that.

Interestingly, I was an extremely shy kid growing up—less so now.

I don't mean to give advice on gaming the system—I'm sorry if I came across that way. The point I was trying to make was much closer to your first line—that the current interviewing system rewards confidence and communication skills, and that debate helped me build those. Ironically, I wish I had communicated that better :)

Also, I agree that there are drawbacks to doing debate. For one, there's a certain "debate kid" mentality that I picked up and had to unlearn.

Thank you for clarifying and confirming my other fear of undesired debate mentality. FYI, it's a good idea to edit your HN profile. When you write good comments, people may check it.

Thank you for the feedback—genuinely. I always appreciate feedback given in good faith.

What kind of edits do you have in mind? I somewhat-deliberately kept my profile vague, but I'm definitely open to suggestions on how to improve it.

I concur. I did BP for only 1 semester in college but the experience proved to be valuable down the line. It forces you to articulate yourself "on the fly", streaming out ideas as they are formed (from core principles / bullet points you have in your mind).

Yes, exactly!

I'm a ex-FANG dev that started a recruiting company, so I am VERY familiar with the arbitrary tech interview process. I'm a huge proponent of contract-to-hire. Even started a company* to make it easier for companies to offer contract-to-hire.

Now before you dismiss the idea because you are "too good" to do C2H hear me out. Everyone knows that actually working with someone on the job is the only way to get an accurate evaluation of an engineer. Everything we currently do in technical interviews (whiteboard coding, pair programming, take home assignments, etc) are just attempts to recreate a working environment. If you are hiring a contractor first, you can do a light-weight interview process and then just bring them on and start working together.

As the dev being brought on you can prove your value before you negotiate your salary. You can also try out the company and manager before committing to an offer.

It's crazy, I know, but I think it would drive dev salaries up if it weren't so hard to change jobs because of the tech interview gauntlet we've created for ourselves.

[*] https://www.facet.net

Employers abuse contract-to-hire to get temporary employees with no intention of hiring.

Some companies might do this, but this hasn't been our experience. It's the opposite. Companies VASTLY prefer full-time employees for software developers because it is so costly when they leave. They want devs to stay forever. They almost always convert contractors to FTE within the first couple fo months. They want to get the golden hand-cuffs in place ASAP.

> Companies VASTLY prefer full-time employees for software developers because it is so costly when they leave.

Companies whose core business is software development probably do, some other companies have almost entirely outsourced development, even where they have very large ongoing custom development needs.

Counterpoint: I used to be a contractor and took C2H positions to see if I wanted to work with these people long term. Most of the time, I was happy to see the door when the contract ended, though all of them offered a conversion.

There was one where I converted to full time, and then we got hit with the dot com crash, so that sucked.

That seems like a bad deal for the company. Even though the person is a contractor, you just trained and ramped them up for the last few months. Now, when they are most productive, you are going to let them go, and replace them with somebody who you will have to train and ramp up all over again.

why wouldn't they just hire contractors who don't expect to ever be hired?

Are you offering insurance to every C2H prospect? Otherwise, this seems like it basically works for healthy people without families.

We provide business insurance coverage for anyone on contract through Facet. For health insurance, we recommend people just use COBRA to keep the insurance they had at their last company. You have to pay for it out of your own pocket, but you just build that in to your hourly rate that you charge.

If I was told that from a company I was looking at, I would drop them like a hot rock. Health insurance is not optional in the US, and COBRA is a giant ass-pain. This really sounds like very little benefit to me for a whole lot of benefit for the hiring company.

So a normal scenario would be `hourly_rate + (monthly_cobra_cost / 160)?`

Why on earth would I quit my job or take days(?) off just so I can apply for a contract position? This would be strictly for unemployed software engineers, or truly desperate ones that needed to get out of their current job ASAP.

Yeah, that's the part that sounds crazy. The benefit is you get to try them out first and you can negotiate a higher salary after you become indispensible. If it doesn't work out, you move on to the next C2H.

Right, that doesn't sound like much of a benefit to me, it sounds like a negatives. I don't think the best caliber engineers would take an offer like this, this sounds like, as I said, an option for unemployed engineers or desperate ones.

Especially in this environment, it could be weeks or months before the next C2H, especially with so many more engineers being laid off. There could be a ton of people competing for a single job.

Also, negotiated a higher salary when THEY hold the cards because now you need to find a new job? That's insane.

Yeah I dunno about that... a long (or even short) list of 6-12 month C2H gigs that didn't turn into full-time jobs screams trouble -- either you're a jerk, or you're bad at the tech.

If you're consistently swinging from contract to contract you're a freelancer, and you should be arranging contracts accordingly, not letting headhunters/recruiters control the playbook by dangling full-time employment.

I agree with the grandparent: C2H usually means you're unemployed, or you want out of your current gig badly, or you're aiming slightly higher than you're qualified; if you're qualified for the role and already have a job then they should be willing to take you on from the get-go.

Yea, I'm in the same boat. Absolutely no reason for me to leave a stable job (even if I don't love it) for a C2H position. There is literally zero benefit to me as an employee to do C2H.

> It's crazy, I know, but I think it would drive dev salaries up if it weren't so hard to change jobs because of the tech interview gauntlet we've created for ourselves.

How does C2H help me jump jobs as a dev who is already employed?

Since I already work 9-5, typically my only free days are my weekends. I might be willing to work for a few weekends if it would lead to a hire, but are those really compatible with what the hiring company wants? My guess is that they want me to take the step of leaving my current job, which comes with loss of health and dental insurance, which is often a big concern.

What you do is you use COBRA to keep the insurance you currently have and just build in the cost to your hourly rate. Companies expect top pay more per hour for a contractor than a salaried employee.

So you are paying for your health insurance yourself, but you keep the same plan.

There's still the risk of them not bringing you on at the end of the contract or terminating it early if it's not going well. But I think the upside of easier interviews and trying out a team out weighs the downside.

There's a class problem with C2H in almost all companies. Contractors are always seen as less than by management and especially hiring committees. I work at a company that I've been hired thru C2H. Everyone on the team, my manager knew I was one of the strongest programmers on the team. But when it came for my conversion interview I bombed. It was run by 3 of the most senior principles with the lowest acceptance rates and a penchant for "protecting" the company from bad hires. Being told despite amazingly good performance compared to peers you still aren't good enough is a nice little gut punch.

A short time later, I did another conversion interview, got people who were willing to tell me "look, we know you can code" because they had SEEN me code and design systems they were using every day. They still did some interview questions, but lo and behold I didn't feel like jumping out the window every second.

Interesting. I haven't thought too much about C2H in particular, but it seems like it might just kick the can down the road instead of solving the root problem; The root problem is human nature.

I would think the idea of C2H is instead of having an intense interview you have a performance review and talk about how you work together. If you still have to pass a high pressure interview attempt to filter you out after working with a team for 3 months there’s no advantage to C2H.

If you /didn't/ have to pass through the same meatgrinder that every other employee did there's a reasonable argument that it could be seen as an opportunity for favoritism. But that sort of ignores that if an interviewer wants an interview to go well (because they feel more empathy for one interviewee over another for instance...) it tends to go better. We like to pretend that "a question is a question" but we're just testing the person's state of mind. I've had interview questions I've answered >10 times and drawn complete blanks because I'm trapped in a tiny room with a person with an inflated title and ego, and their shadow who is intently staring at every movement I make.

Who cares about favoritism vis-à-vis "the meat grinder," if you're just trying to hire good employees?

People who paid a lot of money to get their education and bonafides. Or antifavoritism crusaders that like to tell themselves they can make a biasless interview process.

Just the idea of a "conversion interview" seems nuts. Interviews are a way (albeit imperfect) for you to gauge the likeliness that a candidate would succeed in a position.

If you've already worked with the candidate, you already know this so there's no point in having an interview.

It's not just that they're looked at as less than, it's that they have to be treated as such. I'm on a sort of "party" committee at my work (I know, I know) and we are explicitly told we have to go out of our way to ensure no one on contract gets invited to the events, receives any of the (shitty) gifts that management hands out etc. My understanding of the rule is pretty bad but I've been told it's because they can potentially make a case in court that they were treated as full employees and deserve the same benefits, protections etc.

> because you are "too good" to do C2H hear me out

No I dont want to quit my current role until I have a secure job. I'm not going to c2h to be told six weeks later I'm on the street.

Yea, this guy loves C2H because it makes him a lot of money. It "solves" a problem for employers - which is all he really cares about. Good employers don't use C2H because they recognize talented people have plenty of competing offers for direct-hire.

If I'm quitting my current job to come work for you, I'm not taking on the financial risk for a job that YOU decided needs to be filled, set all of the evaluation and success criteria, and know what your company's culture is actually like.

We make roughly the same whether you go C2H or direct salaried. C2H interviews generally take much less time, so we do have an incentive, but so do the companies and candidates.

You don't have to take a C2H job, but for some people that don't have stellar resumes or are bad at interviews, C2H will give them access to more opportunities.

Right.

For a very good (but not exceptional) developer, I wonder if ending up "out on the street" would be a reasonable expectation if something like this became truly widespread.

If companies like Google were no longer able to filter false positives __at Google scale__ using their current hiring practice, I wonder how long it would take to decide that the next best thing is to contract some tunable number of N contractors for K positions where N >> K and only keep the best M (K <= M << N) of them. (I expect a company like Google to occasionally keep more than K because they can't afford to throw away rockstars if they get a great cohort).

So, even if you're pretty good - if you aren't better than the bottom x% of your cohort (or some other aggregate measure) - you're out. Stack ranking for C2H, basically.

Sorry, but C2H sounds like a great idea to you because you have the luxury of having already earned FAANG money on your side. You'd have to pay me considerable money for me to consider taking a C2H role so I could cover health insurance across the usual suspects (general health, dental, vision, specialists just in case). Like many people outside of this website (and enough here), I don't have the luxury of FAANG money and I live in the United States, so I need health coverage either through my employer or on my own provided I have enough money to offset it.

This is one of the reasons I've always found it paradoxical that people in the US seem to think that universal healthcare would depress the labour market.

Up here in Canada not all procedures are insured, Pharma, Vision & Dental are the big three that most either pay out of pocket or purchase extended insurance for - though there are generally pretty good provincial programs to help with the costs of those. However, nobody is walking around in Canada concerned that they may get fired and then be bankrupted by a heart attack.

Companies absolutely do still bear a significant per-employee cost to support provincial healthcare and residents do need to pay a pretty light fee into a pool (with options available for financial assistance).

The end result is that companies do end up bearing pretty heavy burdens on a per-employee basis - but employees don't end up with as much of a burden forcing them to remain in undesireable employment positions, though some guaranteed income would really free up employees to have a more equal amount of power in the job market.

I’ll point out that untangling health insurance from employment does not necessarily require a “Medicare for all”-style universal care system. This is a subtext to your comment since you jumped there from the parent’s complaint about employment-based insurance.

> However, nobody is walking around in Canada concerned that they may get fired and then be bankrupted by a heart attack.

Mate when I (in the US) took a month off because I got COVID, my insurance tried to cancel. Fucking insane that we have to deal with this shit.

Many consultant/contractor firms esp for programmers provider insurance. Its usually not as good as a top tier company benefit package of course. The biggest issue for me is you're unlikely to get a 401k or tax-benefited investment vehicle, which means if you're making programmer money you can't even do a ROTH or a traditional IRA due to income restrictions. So make sure its a really good jump in pay due to that at minimum.

You can still make contributions to a Traditional IRA [0], you just won't be able to make deductions above a certain income if your employer offers a retirement plan [1].

[0] https://www.fidelity.com/retirement-ira/contribution-limits-... [1] https://www.irs.gov/retirement-plans/ira-deduction-limits

In my experience, with C2H you typically earn more in the contract phase than you would with a base salary, which is helpful in covering benefits or out of pocket expenses. Maybe that's not always the case, but salaried employees usually earn 80% or less what their contract counterparts do.

And you kind of have to bomb the contract phase to not be given the offer, unless there are other circumstances at play (budgets, pandemics, etc). Hang with the team at lunch or after work if they do that, make code contributions asap, and be the kind of person any team would want to have.

C2H usually just makes it easier for them to drop you if you're (surprisingly) not a good fit - and after that kind of time and money investment, they definitely don't want to do that.

> Maybe that's not always the case, but salaried employees usually earn 80% or less what their contract counterparts do.

That's because the employer is covering significant costs on behalf of the employe. A fully-loaded FTE typically costs 30% to 40% more than they're base salary.

Contractors need to make more because they're covering all of those things themselves.

I agree, which is part of why C2H isn't as bad of a deal as the parent thread is implying

This is correct. Company pay much more per hour when you are on contract because they don't have to cover FICA, insurance, etc.

Not just health care, most people don't want to leave a steady paycheck for the possibility of no paycheck.

At my company we do contract to hire, but it's only about 4-8 hours of work. The intention isn't to replace your current job, it's to compensate you for doing extra work for our company while we interview you.

When I was hired I did about 1-2 hours of work for 3 days, after my normal job and was hired before even finishing the "project".

I love the idea from the employee perspective! How can I be sure that the employer is right for me? With c2h I can test that.

As a candidate, I'd rather get a non-insignificant RSU grant up front, with a vesting schedule that begins day 1, than play the will-I-actually-be-brought-on-full-time game.

Companies have no incentive to bring you on full time, either. They can lie and tell you that you will at the end of the term, but it'll either be met with "we do not need this position anymore" or continual contract extensions because the market is suddenly unfavorable, there's no headcount, or some other excuse.

In practice, C2H is more like C2S, where the S is for street. As in where you'll be once bigcorp decides they don't want to pay you anymore.

I would consider C2H if you were willing to offer three months full severance pay and health insurance in the event that you decide to lay me off at the end of the contract.

There is something to this idea, but the discount on the first few months of work as an opportunity to "prove your value" doesn't make any sense. If your value isn't sufficient, they shouldn't renew or take the next step. The C2H process already excludes the value of many valuable benefits.

You're actually paid more cash during the contract period because you have additional expenses. You wouldn't be working for less.

However, it is likely you'll be able to negotiate a higher base salary when you convert than you would have been able to negotiate if you had hired on directly as a salaried employee, because it would cause them extreme pain to try and replace you.

How does C2H get rid of interviews if there are more candidates than roles to fill? Surely you need to filter through anyway. At that point, C2H has the downsides of a permanent position (having to pass an interview) without the benefits of stability, health insurance, holidays, ...

Besides, from the hiring company perspective, only unemployed people or people who really hate their jobs would end up sending their CVs. The competent people who are currently happy where they are would never try C2H. So they are missing a lot of potential good candidates that way.

A number of the systemic issues with contract to hire have already been raised, but here’s another: taking a C2H job can immediately render you ineligible for unemployment, as you are now self employed. If the contract doesn’t work out, you could potentially have lost eligibility for not just unemployment benefits, but related food and health benefits. That’s particularly relevant now with the souped up pandemic benefits.

I don’t disagree that it would be nice if every candidate could take the time to try a company out before they join them, but the real world really can make that impractical. Some of the suggestions this thread makes around “buying” ones way out of these problems (such as knowing to have paid for COBRA ahead of time and being able to afford it) represent knowledge and opportunity that is all together distinct from ones ability to do the job.

This C2H approach seems to ignore the fact that companies have a genuine preference for paying with stock options that they can pay for by issuing new stock rather than expending cash on hand in the US, something which can be a scarce resource at times given a combination of cash flow situations and US tax law. Stock options on short-term contracts tend to be less feasible.

My favorite interview experience of all-time was essentially a two day contract. The company listed a couple of simple tasks in their codebase that didn't need a lot of context to complete. And there was no pressure to complete all of them — it was simply, pick a couple that you think you can accomplish over two days.

I put up a few PRs, then a final assessment was a discussion that was essentially, "Now that you've seen our codebase, what are your thoughts on how we could improve our architecture, etc.?"

I was paid for my time as well (because I was legally able to accept payment at the time). I think that's actually a larger hurdle — companies that disallow contract work that prevent engineers from getting paid for this type of interviewing, although I suppose it's in their best interest.

this is great and may work for some companies and candidates. hard to scale to FAANGs

C2H is terrible for the employee and you should know that, as a professional headhunter.

Negotiating your salary upfront is likely going to give you the best deal you can get. Don't kid yourself into thinking that you can "show your value" to improve your negotiating position. Your first few months on the job are usually when you are at your lowest performance, since you are learning the company's processes. That is your trial period.

You are in your best negotiating position when the company needs you more than you need them, and that is absolutely NOT after doing contract work for them for a while.

Once you're at the point of converting, if you get a lowball offer, the contract position is a black mark on your resume that hurts your negotiating position with other firms.

That said, I would be happy to do C2H if the following conditions were met:

* Signing bonus paid in full at point of signing the contract, with no clawback.

* Full health coverage during the contract portion (extra money plus the option to buy from their plans).

* Severance pay at full salary for at least 6 months if you choose not to convert.

* Non-exclusivity during the contract period (I can do other work if I want, and you don't own any IP I produce other than what I produce explicitly for you).

I think this is sufficient to equalize the negotiating disadvantage that I get for taking the contract. I don't think any of your clients would go for it.

That's not what we've seen. For example, one of our customers raised salary bands across their entire software division to accommodate the salary demands of the contractors they wanted to convert to salaried.

In most cases companies have a ball park salary in mind for the position, so when it comes time to convert there is some room for negotiation, but no surprises.

Companies are so desperate for good software engineers that they aren't going to let a good dev go after several months on contract in order to low-ball you and save a few thousand dollars.

Sorry, your pitch to clients is "throw away all of your leverage (existing job, unemployment qualification, health insurance, etc) to contract for a few months and then you'll get a great fulltime offer"? That's...deeply implausible.

I'm almost willing to believe this works for people who absolutely are unable to pass any sort of interview, but in that case aren't they still in trouble? How does a company decide who to bring on for the contract? Surely you can't replace the interview process with this.

>> Negotiating your salary upfront is likely going to give you the best deal you can get. Don't kid yourself into thinking that you can "show your value" to improve your negotiating position. Your first few months on the job are usually when you are at your lowest performance, since you are learning the company's processes. That is your trial period.

That does not conform to my experience as employee or as an employer. Do you have any evidence to support it? If does sound like something a recruiter who wants to take a cut of your first year salary (from the employer) would say.

How does this work with non-local applicants?

I would certainly move, even internationally, for the right job, but I'm not going to relocate for a shot at a job.

Yeah, I wouldn't relocate for a C2H job either, unless it was a dream job or something. I think in those cases it would be best to opt for salaried and just deal with the current technical interview process.

Everybody does not in fact know that "actually working with someone" is the "only" way to get an accurate evaluation of an engineer. There are recruiting/qualification disciplines besides temp-to-hire and whiteboard interviews.

Temp-to-hire is problematic for its own distinctive reasons:

* It's difficult to generate apples-to-apples comparisons of candidates in a temp-to-hire setting because the engineering workload isn't constant; candidates are getting the luck of the draw with whatever the tasks are during their contracting period.

* There's as much subjectivity built into the projects candidates get during their temp-to-hire as there are in whiteboard interviews, so culturally preferred candidates are just as likely to get fast-tracked in a temp-to-hire process as with interviews.

* Interview processes routinely evaluate many candidates for a single opening --- that's expected in basically all hiring, across every industry --- and can't reasonably offer temp roles many candidates simultaneously, so in effect temp-to-hire just pushes the qualification problem to a higher level of the funnel, where the recruiting process decides who to extend temp offers to.

* Temporary contracts draw out the hiring process and potentially make it take far longer to finally fill a role than other processes do; you're required to trade accuracy of evaluation against the time taken to fill a role, bearing in mind that when candidates wash out of their temp contract, you likely have to start all over again with another candidate, making it even more important to pre-screen candidates before offering roles, which just recapitulates the whole problem.

* The tech industry is notoriously bad at evaluating on-the-job performance too; every developer I've talked to has stories of dead weight team members† that were kept on the team interminably because no process existed to evaluate their performance accurately. There's a whole cottage industry of people who have failed up† through role after role, because there's no resume difference between people who phoned in a role† for a year or two and then decided to move on, and people who did well.

* Perhaps most importantly, temp-to-hire is deeply unfair to candidates; a career norm in tech is that people secure their next role while winding up their previous one (in fact, that's a norm in most white collar industries, which is why there's so much written career advice about how to handle counteroffers). Taking a temp-to-hire position requires a candidate to forego income security for a chance at a job; when that job doesn't work out, they're now in a distressed position when looking for their next job.

(I'd add, though it's a nit, that temp-to-hire positions virtually never compensate appropriately; contractor wages are routinely 2x+ what FTE wages are, in large part precisely because of what makes temp-to-hire so attractive: freedom to fire on a moment's notice with no commercial/reputational cost.)

† Shut up, Kurt

On principal, I don't think "temp-to-hire" is the same thing as contract-to-hire. With a contract to hire roles, you are filling a full-time position on your team with someone that is paid differently. They are treated as a new hire.

I wouldn't take a "temp-to-hire" or a job with a trial period either.

I can’t imagine how anyone would give up their employer-provided health care (which is all we have in the US) during a pandemic to go into a contract-to-hire scheme. That might have made sense last year but it definitely does not now.

I'm not gonna quit a good job to take a C2H job no matter how good it is, especially in the crazy times.

Huh, the only successful interviews I have are the ones where I'm burned out after a long, fruitless, frustrating job search. And so I walk in assuming the interview will be a waste of time and just not caring anymore.

That's when I get an offer.

People can smell desperation, and rightly or wrongly, it pushes them away.

Haha. So the more driven and hungry i am the worse ill be viewed. The more silver spoon and apathetic ill be cherished.

Sounds like dating ;)

A different way to view this is that being desperate makes you behave badly. For example, someone desperate for a job won't push back against bad decisions and will become a "Yes Man". Someone desperate for a date won't engage with the other person as a person.

That is a different way to view it. An incredibly negative seeming view of humans. Ill lay my cards down though, i have limited years of really varied experience (ups, ski resort, landscaper, metal grinder, fork lift driver, house painter, door to door vacuum sales, google pm, msft Analyst, trying to break into infosec.)

In my short list (ignoring years of time wasted travelling), i have only met 1 yes man. He came from money, and sales. Owned 4 large businesses, and would over promise, under deliver and over receive ALWAYS.

So what are your experiences like and from where?

I think no backbone can be a personality trait, sure. But it is NOT a trait of 'need' and action, in my experience.

Saying "Yes man" probably derailed my thesis since it's associated with the higher rungs of the corporate ladder. I thought that this would be uncontroversial.

I'd like to focus on Dating because I disagree with your offhand comment, even though it was in jest and is probably more cynical than you are in person.

> The more … apathetic ill be cherished. Sounds like dating ;)

Feeling bad makes your behavior suffer. Being angry makes you less nice to people. Being tired means you don't engage with people. Being scared or anxious makes you hide your vulnerabilities and close yourself off. I'm not saying that badly behaved people deserve to be ignored, I'm acknowledging that it is uncomfortable to interact with people who are hurting.

Being "driven and hungry" while dating makes you behave worse. It makes you over-eager to please the other person, and it flattens out your personality. Instead of focusing on making the person like you, you should be exploring whether you like each other. A date can be successful even if the two of you don't end up dating because you two would not make a good couple.

This is so true and yet so difficult to control for.

Some of this is like the Groucho Marx joke "I don't want to belong to any club that will have me". For companies, it's "I don't want to hire anyone that doesn't have a job". The thought being that if they were any good they'd have a job already.

This is one of the things I look for when I'm being interviewed. Does the company want me to be desperate?

Some companies will only hire those who are desperate and will do anything. I assume, hopefully correctly, that those are the companies not worth working for, so instead I use the interview opportunity to get better at interviewing. Because these companies want to take advantage of their employees, I do not feel bad wasting their time.

Careful, lads, it's possible to be TOO confident.

I interviewed at a few places for some really easy, fun-looking jobs while I was doing freelancing. I didn't care if I got the jobs, but they did look trivially easy compared to what I'd been doing earlier in my career. Surprisingly though, after making it through all the technical people and talking to the CEO in every case, I never got the job.

Later on I applied to a really awesome job at a FAANG-class megacorp, and the hiring manager's manager called me. He said "Look, you've gotten the highest marks I've ever seen for a candidate. Everyone loved you. But I have a hunch that you aren't going to be dedicated to this job. So before we move forward, I want your personal promise that you aren't planning to just hop ship in 6 months to start a company or something"

I was JUST starting to get nervous about finding a new job at this point. My consulting work had dried up, I was down to about 3 months' worth of cash, and I was thinking about selling my car to get a few more months' runway while I grind Leetcode in order to be able to pass the bar at Google.

So you can imagine how shocked I was to hear that this guy thought his awesome senior engineer FAANG job wasn't good enough for me. I promised to stick it out, got the job, and so far it's been my favorite job of all time (other than the glorious week where my crypto trading bot was working well and just printing free money..)

I had a similar experience for my current job. I went to the interview on whim, assuming that I would fail horribly. It was only practice.

Then my brilliant plan was spoiled.

This was true for me, too.

My previous startup I worked at had gone under, and I was thinking about starting my own. An old coworker of mine invited me to lunch. We had a couple of margaritas, and he asked if I wanted to see his office.

I went to check it out, and suddenly he asks if I can talk to some of his coworkers.

So I basically interviewed while drunk and not fully realizing I was being interviewed.

They offered me the job, and I have been working there for 8 years now.

Maybe the best way to be relaxed during a job interview is to not even realize it is happening. And having a couple of margaritas can't hurt.

This is described in http://carlos.bueno.org/2014/06/mirrortocracy.html

Some companies stress interpersonal fit over technical ability, as you are initiated in "the Culture."

I actually agree.

I enjoy telling this story, because I think it is funny and always gets a laugh.

I do think, though, that it is indicative of a problematic culture in tech that perpetuates hiring people who are like you. I got the job when I was a young, single guy who could fit into that sort of 'brogrammer' culture. I am lucky in that the company has changed over the years (bought by a large corporation, and we have mostly moved on from this sort of culture), because I have two young kids now and can't live that sort of life anymore.

What is worse, is there are a lot of great software developers who NEVER could fit in to that sort of culture, and they would have had a lot harder time being hired.

That sounds incredible.

I'd like to see this two margarita interview technique catch on.

That is funny as that is the way alot of my interviews go as well. I tend to have extreme anxiety on a job I really want and "pysch myself out" so to speak. I really dislike typical tech interviews as it is alot of pop quiz obscure answers under extremely tight timelines never seen on the job. I have had interviews where 3 interviewers are all demanding answers at the same time and it was horrible. I hope at some point in the future this is realized as a colossal fail in the industry and a new method is realized.

That's how non-FAANG goes. I work at an old, non-tech corp and the interview (just one) was basically a popularity contest with some light technical questions. Don't screw up and crack some jokes and you're in. Of course, that means our level of technical ability is low, and we have poor project outcomes because of it. But it's a nice place to work.

You must be coming off as confident subconsciously with your air of indifference you carry for the interview process.

Same. Sometimes I only go on an interview to "practice", and BAM! Offer comes.

That can help a lot with anxiety. If you think it's pointless and you'll never get the job, you have a chance to just have fun with it and do what you can do.

My first time interviewing with Google (in 2008), the press was reporting that they were in a hiring freeze, I had no big names on my resume, so I figured it was basically hopeless and I'd just do the best I could and enjoy my trip to Silicon Valley. After my interview, I was like "Well, I did okay, but you really have to be perfect to work at Google and I wasn't perfect. I'm never coming back to California anyway, so I might as well do all the tourist stuff [I'd held an extra day to interview with Twitter, but they ended up not moving forwards with my candidacy] and just treat it as an all-expenses paid trip to SF." So I just went to Alcatraz and Fisherman's Wharf, treated myself to a nice dinner, and took the red-eye home to Boston. As I'm groggily waking up after a post-red-eye nap, the Google recruiter called me to say "Your interview feedback looks great. We'd like to fast-track your application on the assumption that the hiring committee will say yes."

Been living in California for 11 years now, and I just started back at Google after working there for 5.5 years, leaving, and doing startups for another 6. The most recent round of interviews was the same - I was super nervous for Lyft, Stripe, Coinbase, Netflix, etc. and ended up not getting the job, but with Google it was like "Hey, I worked there once, I know people who also know my interviewers, I used to give interviews like the one I'm sitting in now, what's the big deal?"

I also treated my Google interview as a free trip to the Bay Area, and used it to visit a friend and do touristy stuff. Wasn't expecting an offer, but got one and stayed there for many years.

I've read advice that you shouldn't schedule your interview on a Friday, but that's exactly what I did so that I could spend the following weekend in SF :)

wait so Google did the entire red black tree interview thing again, even though you already worked for them for 6 years?

lol this industry is hopeless

Yea, the "interview" should have been nothing more than a review of past performance appraisals and a conversation to assess motivation/goals/etc.

Pretty much. If you leave and come back at the same level within 2 years they skip it, but I'd been gone for 6 years, and I was coming back 1-2 levels higher (part of the point of interviewing was to determine level). They want to determine if you a.) forgot how to code, which is entirely possible if you've been doing other stuff for a while and b.) have learned something new & valuable that warrants a promotion.

The panel was different, too - my first time around I had 2 algorithm, 2 UI (Javascript), and 1 system design interviews, while the second time was 2 algorithm, 1 behavioral/leadership, and 2 system design interviews. This is probably reflective of the differing role and higher level.

What’s frustrating to me is clearly you’ve spent 10+ years on the industry, worked at big cos like Google and startups, passed Google onsite twice, and still got rejected by all those companies. Which would be fine if industry execs weren’t constantly bemoaning about talent shortages and difficulty hiring engineers. They’ve established a cultural pattern of talking about how acceptable it is to reject highly qualified engineers, labeling them “false negatives”, then refuse to introspect on that when discussing why they find hiring challenging. (I’ve also spent 10+ years in industry, 3+ years as a Google SWE, and got zero feedback rejections from companies like Stripe and many many others in the last few years.)

I had a quick phone screen with a recruiter the other day. For unreleated reasons the night before I hadn't slept much. That morning I was clambering about in the woods and stumbled on a wasp nest and got about 15 stings all over my body. I took 3 Benadryl anti-histamines before I remembered that I had the interview that afternoon. I was a complete space cadet.

Somehow the recruiter was still interested.

Maybe this should be my strategy for every job application? :-)

I solved this horrible problem with legal anxiolytics. I crashed and burned every interview I had until I decided to drink kava[0] beforehand until I couldn't care less about whether I got the job. I've been working there for years now.

Like it or not, interviews are a game. Smart people will figure out how to play them after losing enough, though maybe some of us take a bit longer ...

[0] https://en.wikipedia.org/wiki/Kava

Similarly, I used to find I interviewed better when I had a hangover.

(I'm older now, have hangovers less often, and find them much more tiring, so this might not apply any more)

How I got my internship that converted into a full time job and everything that followed from it. I literally gave the interview and went to the sports stadium. To this day I've been job hopping without interviews. Everyone who works with me knows my skills (and my drawbacks) and they're the ones who hire me or recommend highly of me to lead to a job. My next job may not be so lucky so I'm starting to dust off my "interview skills". It's frustrating. I'd rather be building something usual than solving these. But that's the nature of it today.

I got my first salary job this way. After that I did my typical analysis seeing if I couldn't learn from the situation and improve beyond it.

My conclusion was if I interview to gain interview experience, not to get the job, I do far better, and I gain interview experience. This is the local maxima I found from that, but there could be better.

+1. Either this, or I have an offer in hand and so I don't have to care anymore.

The role of anxiety in interviewing became very clear to me during a season when I failed all my interviews until I got my first (and a very good) offer. Suddenly I started passing all of them. It was night and day. I was very surprised having always had much longer and frustrating recruiting processes.

I'll sometimes go on interviews when I'm NOT looking for a job. It's good practice and keeps you informed of what's out there.

Here's the study: http://chrisparnin.me/pdf/stress_FSE_20.pdf

They asked students to find the "Longest Substring Without Repeating Characters".

For the formats: "We created two interview settings: public setting and private setting (see Figure 1). In the public setting, wearing an eye-tracker, participants solved the technical interview problem in the presence of an experimenter. The participant was instructed to talk-aloud while solving the task, which is a standard practice in technical interviews [28, 47]. The lab space contained a whiteboard at the front of the room. An experimenter was situated near the participant for the entire session without interrupting participants’ thought process. If a participant asked for clarifica- tion, the response was brief. For example, if a participant asked, “Does this look correct”, the experimenter was instructed to reply, “Complete to the best of your ability”.

In the private setting, participants solved the technical interview problem in isolation. Participant were provided a private room with a whiteboard. Participants were informed that the experimenter will step out the room and will not return while they are solving the problem. Thus, they had to make sure that there is no questions left for them. When the participant was ready to begin the task, they wear the eye-trackers, the door was closed, and the participant worked in privacy, until the task was completed within allotted time. After the experiment, we clarified with participants if they had any uncertainty about the problem they just solved."

They ran a randomized controlled trial on the effect of having another person in the room, which is interesting. It does not, however, show that these interviews fail to assess software skills.

This is also essentially the worst case for having another person in the room: they are completely superfluous, just watching you, not helping, not communicating, not collaborating. Not even saying things that might make you more at ease.

> They ran a randomized controlled trial on the effect of having another person in the room, which is interesting. It does not, however, show that these interviews fail to assess software skills.

The headline of the linked article is indeed heavily editorialized. There were differences between the two groups, but the study didn't conclude that the interview was only assessing anxiety.

> This is also essentially the worst case for having another person in the room: they are completely superfluous, just watching you, not helping, not communicating, not collaborating. Not even saying things that might make you more at ease.

That stood out to me, too. In an attempt to remove the influence of the observer from the equation, they simply put a silent observer standing over the person's shoulder. In the example photo, they have an observer standing almost literally over the person's shoulder, silently, observing their every move. It's not clear if that's how the experiment was conducted or if it's just an example photo, but it's not how any real world whiteboard interviews are performed.

Even the study participants noted how strange the situation was:

"P25 felt unnerved that someone was “watching me and they are not commenting on my progress”

Wearing an eye tracker creates one hell of a different context.

Being ask to solve a problem without your typical tools does as well.

I assume these tests don't actually get to the root of the issue but only represent a portion of behaviour.

The best technical interview I've had:

- Take home: 2 days to read a newly released Deep Learning paper.

- On site: explain the paper to the hiring manager.

- Evaluation: (1) Did the candidate understand it? (2) Did the candidate go farther, and look into the cited papers and understand that? (3) Can they explain Deep Learning concepts and talk their way through related problems?

In 5 months of searching for a job, this felt like the most relaxed interview, because it was really a conversation between people in the field. It was also the most demanding and interesting job I encountered in these five months. I got the job, but this happened especially because the manager actually did the work of respecting my experience and talking to me, instead of going for the traditional and "safe" BS.

An opposite experience:

Technical Interviewer asks me whiteboard SQL question. I answer it, only to be corrected on a minor point. Turns out, when I tried it out on my own, I was actually correct. This person was to be my manager.

I did get an offer from them, but this and other red flags made me decline.

Takeaway for managers who actually care about hiring well:

- Don't force your candidate through one off tasks that don't represent their day to day. Is your Data Scientist really just running SQL queries and optimizing computational complexity of their algorithms? Isn't the main point that they can think critically?

- Hasn't their previous experience shown they can code? Their Github? Sure, give them a (quick) take home project if you really want to make sure. In the rare case they faked their way through the whole thing and you really couldn't tell after talking to them for a few hours, you will surely notice in the first few weeks.

> Isn't the main point that they can think critically?

Sadly, I'm coming to the conclusion that this is at best completely orthogonal to anything most people care about, and at worst, it's a liability.

That's a great idea! Was it for an MLE job?

Data Scientist, but would be easy to adapt.

The interesting thing is that Leetcode style problems encourage your code to go into the opposite direction that most production code should go. The game of Leetcode is to write the most consise, clever function that solves the problem in a tricky way. Thats exactly the kind of code I would flag in a review - sure it might be super efficient and elegant, but good luck maintaining it a few months later. I would way rather go for a solution thats less efficient but easy to maintain - I can always improve it later if it becomes a bottleneck (it usually doesn't).

I know the take home assignment is a divisive subject, but I think it's the perfect way to evaluate a person's coding ability. Did they use dependency injection or did they reference globals straight in the code? Did they break down the code into small testable chunks or write one giant file? Would their code pass code review? These things are way more important for most codebases than shaving an order of magnitude of big 0 complexity.

The only issue with the take homes is that they take a ton of time. Most companies say that it should only take a few hours but also ask for production grade code. Thats impossible - clean code takes time. Companies should at least start compensating for the time taken to do the assignment.

My favorite interview process involved a take home assignment which if you pass takes you directly to the final onsite interview. There the live coding sessions involved building more features onto the take home. As an experienced programmer who sucks at Leetcode I really hope the industry moves in this direction.

This is an accurate view of take home assignments. Most of the take home assignments I've done were "supposed" to take 5 hours. Most of them too around 20.

The worst take home assignment I ever had was an extremely tricky brain teaser, the answer to which could not be found on google, and had to be done in the not very popular language this company used. So I not only had to write useless brain teaser code, but I couldn't do it in a language I was already comfortable with.

Yea, I had one take home problem where the instructions stated "Please don't take more than 1 hour to solve." So, just once, I decided to hold myself to one hour, and barely managed to get to a solution that worked.

During the code review in the follow-up interview, I was raked over the coals for not writing tests, failing to cover edge cases, etc. ,etc. Basically, they expected production-ready code. Didn't get the job.

And that's the worst part about the take home assignments, all the people you're competing with are claiming they did it within the time limit when they're really taking far longer to make it perfect.

I’ve shared the same experience! Coming out of college every time I’d get a take home I would always blow way over the expected allotted time. They’d say it should take 5 hours and I’d end up on it for 20 since I actually had the time then.

Year’s later I did the same as you and decided to stick to the recommended time, because I really don’t want to waste my nights/weekends on this junk anymore. Working full time there’s barely any time for relaxing anyway. Sure enough, the same thing happened in review. Got absolutely shredded on my implementation. Didn’t get an offer and kind of wish they didn’t even call me back.

I like the data science interview process a lot more, though I'm biased enjoying the kind of work I do.

It's somewhat common during a data science interview to be given the actual problem you would be working on if you took the job. Often times you're given multiple days as a take home or multiple interviews to go through it, because the problem is difficult enough it does take research to begin to hypothesize a possible solution.

Most software engineers would cry murder if you gave them a 3 to 5 day take home interview that is an actual problem the company is trying to solve. "It's free labor. They're taking advantage of you." but in the data science world solving a problem can take 3 to 12 months, not 3 to 5 days, so you can't solve it in that time frame, just come up with ideas that lead to paths forward. The company can't take advantage of you this way.

On the SWE side, a near equivalent would be pairing with someone on the actual code base with an actual jira issue for a day, learning the code base, seeing the problems in the bug tracker, and then deciding from there if those are the kinds of problems you want to spend the next handful of years of your life on.

I like this kind of problem because I know what my 9 to 5 will be for the next year to years and if it is what I want to be doing with my life.

> Most software engineers would cry murder if you gave them a 3 to 5 day take home interview that is an actual problem the company is trying to solve.

So pay them. $400 for 10 hours of work is nothing for a company compared to the man hours spent on the hiring process. It also signifies to the developer that this company is serious about hiring them.

It's ironic that people who want the job will be nervous and might interview poorly but people that are only interviewing for experience or just don't care will do better.

My best interviews are definitely when I was already happily employed or when I already had offers. The confidence boost is huge.

Definitely my experience. If I don't really care about the job either way I do quite well. If I really want or need the job I generally bomb the interview. So often I've ended up with jobs that I didn't care a lot about.

My Mom makes this point about nerves. In the last century when she was interviewing she got lost and showed up 45 minutes late to the interview. Thinking she had no shot, she was relaxed and interviewed well and was hired.

When she left that job 10 years later, her boss noted, that "one of the things she learned from her was that it was ok to hire someone who shows up 45 minutes late to their interview.."

This is an interesting anecdote. All too often, interviewers seem to be looking for any reason they can not to hire someone rather than looking for evidence that the person would be a good co-worker. I am guilty of this to some extent, but a lot of it comes down to the interview culture/framework of the company in question.

I think I'm the complete opposite. I want to hire you if you come in to interview. Interviewing for an open position takes up so much time from so many people during the process, why would I want to spend more time than necessary. When I'm in the interview I try to do everything I can to make sure you can succeed and looking for reasons to hire you, otherwise I have to spend more time interviewing.

Neediness is always perceived as weakness. Same with negotiations, the less you need something, the stronger position you have, the better deal you get.

I used to work in a client facing role for enterprise software before I got into programming. The ability to stay cool and override nerves with sheer adrenaline that I developed by attending high pressure client meetings is 1000x more helpful to me in technical interviews than any leetcode practice problems. If you have the basic technical knowledge and problem solving skills then 99% of the interview is just staying calm enough to use them properly without coming off like a nervous mess. Much easier said than done, I know.

Anecdotally this has been true for me. For jobs I didn't care about for whatever reason I have a much high success rate than for jobs I really wanted.

I guess it's possible that the jobs I wanted are at more prestigious companies and therefore are more picky and the interview is just harder to pass. But I suspect it's not just that, and psychology plays a part too.

This is likely a source of gender disparity in technology. Men tend to be more confident in math and engineering situations, even when they know as much or less than their female peers (the much-reported "gender confidence gap")[1][2][3]. We are disadvantaging equally capable women by using an interview format they are less likely to do well on.

[1] https://onlinelibrary.wiley.com/doi/full/10.1111/j.1949-8594...

[2] https://eric.ed.gov/?id=ED542116

[3] https://www.theatlantic.com/magazine/archive/2014/05/the-con...

I've been interviewing SRE candidates for a long time. In theory, being able to make technical decisions under high pressure is integral to the job.

But interview stress is very different than work stress. It doesn't seem to translate well. Some people do fine during the interview and then crack under real pressure, and some people it's the exact opposite.

I've changed my interviewing to involve very little coding. I might do FizzBuzz on the phone screen just to weed out the folks who really can't code anything on the fly, but that's about it.

After that, most of my interview is me explaining some real works tasks that they would have to perform, and then asking them how they would try finding the solution and if they have ever solved a similar problem in the past, and if so, how. I also ask them about difficult problems they've solved, how they solved it, how they found the problem, etc.

My goal is twofold -- try to figure out how they research problems they don't know the answer to, how they investigate issues; and give them lots of examples of what will be expected so that they can bow themselves out of the process if they don't think it's a good fit for them.

This obviously works much better with people who already have a job, and are more likely to take themselves out of the process, but so far it's worked pretty well.

This is what I do too. I try to engage the candidate in what they have worked on. Then I try to adapt problems to their environment and have them explain to me how they solved them. Or if were to encounter that problem in their environment, how would they solve them.

Granted, you need to be able to program. But being able to recall and implement particular algorithms from memory isn't particularly valuable to the job. I want problem solvers. People who have dealt with complicated problems and found solutions. Or at least know how to find them.

I've observed that candidates hamstring themselves in ways that seem to me bizarre. Indeed the interview is an unfamiliar situation, so why would you do something like use a language other than one you know well, or code in an environment you don't feel comfortable in? I have had those happen in interviews I conduct so many times! This is for the "front-door" interview, not the on site.

Somewhere around halfway through, when the candidate is floundering and failing on an objectively quite simple problem (I promise it really is that simple), they'll say something like "sorry, I would usually have no trouble doing this in Scala". At the outset I give the option to use their own setup or Coderpad, either one works; they'll choose Coderpad and then halfway through say something about wishing they had such-and-such from their preferred IDE.

I think there is some psychology at play here, where candidates think if they do it on hard mode, it's extra points or something. I really just need them to solve a simple problem so we can be sure they know how to code. The ability to code with both arms tied behind your back is not a signal we're interested in measuring.

Yeah I got that too. I would tell them, "do this in your favorite language" and sometimes when they get stuck they say, "I don't really know Python that well". Then why did you choose it?!

Although my favorite was, "I don't know how to write a loop, my IDE usually does it for me".

That experiment assumes that SWE job is exclusively about solving technical problems, so we can just "lock the candidate in a private room and let them solve it".

In reality human to human interactions, sometimes stressful, are part of the job. Customer & PM requirements, technical debates, giving feedback, explaining your technical solutions, etc. are common.

Maybe part of the problem is the education that does not prepare people to reality of the job.

Note: I do realize that SWE interviews suck, and I'm not trying to defend them. Just making comment on the study.

Agreed. When I'm conducting interviews, the actual code produced is less than half of what I'm looking for.

I want to assess if the candidate can communicate their thoughts effectively, if they can identify alternative approaches and justify their decisions, how they receive and incorporate feedback.

Locking someone in a room to write on a whiteboard is not a coding interview.

Hassling someone once a minute while they try to write algorithmic code is not a great way to assess anything useful either.

I think a hybrid model (create solution in private, then present it and answer questions) might be better. That way you can assess the person's communication without looking over their shoulder.

Hopping aboard your train of thought here... I've never been asked to whiteboard anything truly heinous, and I absolutely despise interviews that are set up as a gauntlet to be ran.

But if I'm interviewing you for a senior SQL engineer position, and you can't talk me through whiteboarding a simple query with one join and one aggregate function... you're not senior material. I'm sorry. We need people that can communicate and explain themselves, and talk through their own line of reasoning, where I work.

Again, I'm not defending SWE interviews... but of course people are better at expressing their skillset when they're in a controlled environment. The trouble is, the workplace isn't a controlled environment. Life isn't a controlled environment.

I don't want to work with brilliant programmers that have zero social skills. If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?

Well, I can communicate and explain things reasonably well to a room full of people. I ran a side business doing that for about seven or eight years.

And I can ship products. I've done that a bunch of times--in several cases I've written most or all of the code in the shipping product.

I can't interview all that well, though, and I'm absolutely terrible at whiteboard coding, or pretty much any kind of problem-solving with people watching me. Except pair programming. Pair programming with a colleague that I know well is no problem.

I have two obstacles. One is anxiety. I score really high on trait neuroticism. It's one of the so-called "Big Five" personality traits. One of the things it means is that I have a lot of self doubt regardless of cause or circumstance, and a lot of anxiety to go with it. As for "working on myself a little bit," the Big Five are extremely resistant to change. Basically, the only things we know of that affect them much are psychedelic drugs and catastrophic trauma. Working on myself a little isn't going to reduce my trait neuroticism much.

I've been through quite a few tech interviews, including some familiar big names. I've gotten offers. But that's never been because of my performance in coding or solving brain teasers, at which my anxiety has been uniformly awful, and so has my performance.

The second obstacle I have is that I cannot combine certain classes of cognitive activity. For example, I cannot navigate and carry on a conversation at the same time. I can drive just fine--safely and competently. I just can't navigate if involved in a conversation, and will predictably get lost. It's predictable enough that when my daughter was a teenager she exploited it for laughs. "Let's see where we end up!"

Another thing I cannot do is solve logic and programming problems while being watched or while carrying on conversations with strangers. The interaction with the strangers forcibly occupies 100% of my attention.

I don't know if I'm representative of a vanishingly small fraction of the population. Maybe so. But maybe there are others more or less like me.

Doesn't mean you need to change your hiring preferences. Probably does mean you won't hire me. Oh well. I seem to be making do.

If there are others like me reading, and wondering what to do about it, one thing that has worked well for me is to do good work for people and cultivate good working relationships with them. I have more often than not been hired by someone who already knew my work, or on the recommendation of someone who knew my work.

I'm sorry to hear that. Every situation is different. If I were in your shoes I'd try to be open about it with recruiters and figure out / propose some alternative that work better for you - whatever that might be. Also keeping good portfolio of Open Source projects, blogposts etc. might help convince potential employers that you're worth considering and that your 'condition' (for a lack of a better word) is real and does not affect your work performance. Keeping good reputation and connections (which you already mentioned) is also very important.

From my experience companies are very flexible and open minded - mostly they care about just not hiring someone that they will have to fire later.

No need for sorrow. Everyone is dealt some advantages and some disadvantages. We work with what we have. I seem to have done all right. My resume is long and I have good stories to tell. I can't complain.

I was responding to another poster who said that "If devs can't muster up the courage to answer interview questions... maybe that means that they should work on themselves a little bit?"

I mean, maybe. Maybe sometimes. Then again, maybe not.

Maybe "working on themselves a little" isn't necessarily the answer. There isn't a known way to "work on myself" to affect my trait neuroticism in any significant way, for example.

Maybe it's not necessarily about "courage". I mean, courage is doing something that frightens you because it needs to be done. I know what that's like. Technical interviews don't even rate. But that doesn't solve the brainteaser on the whiteboard when a stranger's watching me.

In general - working on yourself is usually somewhat possible. For many people public speaking etc. is something they have to practice and they do eventually learn. In some situations though it might not be. All individual cases are different.

So I wouldn't be too hard on myself, but also I wouldn't give up. From what I read, you might have already tried, so oh well, maybe you're just stuck with it.

I like "We work with what we have". That's something I deeply agree with.

That's a good point. But for a lot of people it's not even a problem to do that in front of your co-workers who you work with on a daily basis. Even when I had to work with customers, I had zero problems talking to them, getting requirements, and figuring things out.

If I had to land that customer for the business, I would surely botch it. That's why I didn't go into sales.

I think that the difference is you have much higher stakes in an interview. If you make a mistake with a co-worker or customer, you can usually just go back and fix it. If you make a mistake in your interview, it can cost you the job and it's not so easy to go back and fix.

How right you are particularly when a role has any tie to a production support situation. Not only do you need to solve the problem, but you need to do so when other people are weighing on with their opinions on what might be wrong, escalating when it's not solved fast enough, demanding answers you might not have or asking you to defend the ones you do, particularly if they answers are ones no one wants to hear.

This is the genesis of the infamous behavioral interviews where people ask about conflict resolution skills, how and when you'd escalate problems you can't solve, name a time when you had a problem and how you solved it, etc.

Yes, but real-life situations are rarely as high stakes as interviews, on the individual level.

Sure - you can find yourself in very stressful situations at work, but I have never in my life been in a situation where you're given 45 mins (or hours, for that mater) to solve some problem, or you get fired.

Simply but, there's an incredible discrepancy between the stakes at play, between interviews and work.

To put it a bit more extreme - but to hammer down the point: Imagine working on some problem, with no help whatsoever, and that in front of you, there's some guy pointing a gun against you.

Just imagine how stressful that must be, and how much of your focus is distracted on reading that guy.

Same goes for interviews. For a lot of candidates, that interview is their way out of poverty or lower-class living. And the judge and executioner of your future, is sitting in the same room as you. It's a very stressful situation - probably one of the most stressful situations in your life.

I think that's one of a big problem for candidates to fix - that mindset is wrong and should be changed. An interview should not be a "high stakes situation". It should be more of a casual event - trying to get each other and see if there's any fit. An interview that didn't go well is effectively the same as not having an interview at all. Worst case is losing some time and even then it's often a possibility to learn something.

Taking it all too personally is a big mistake. Sometimes even candidates that did really well don't get a job offer for weird reasons, etc.

It's very similar to dating. Desperation ruins everything.

I think that might be a reason why a lot of people report getting a job after they stopped carrying anymore.

The methodology:

> For this study, researchers conducted technical interviews of 48 computer science undergraduates and graduate students. Half of the study participants were given a conventional technical interview, with an interviewer looking on. The other half of the participants were asked to solve their problem on a whiteboard in a private room. The private interviews did not require study participants to explain their solutions aloud, and had no interviewers looking over their shoulders.

> Researchers measured each study participant’s interview performance by assessing the accuracy and efficiency of each solution.

[...]

> “People who took the traditional interview performed half as well as people that were able to interview in private,”

I appreciate their effort to use science to support the narrative that tech interviews suck, because I do think they suck and need to change, but isn't N=48 a little small? Also, if it was only undergraduates or graduates, who's to say their performance is indicative of the rest of the tech workforce?

The small N is addressed by the authors.

To acquire a larger N would mean recruiting more participants, which would require either compensation/extra credit or rely on people volunteering ~1 hour of their time. Likewise, collecting that data would require time and money. It is an time/resource issue with human research. You can require a small N and get some results or require a large N and not have enough funding to support it.

Also note, what is a "large N"? There is no set in stone amount and I've even seen reviewers say a sample size of 10,000 is too small.

I'm curious as to what makes a good size of N. To me, N=10 would be do small, N=48 seems a little under what is should be, and N=100 seems sufficient, but I don't have a real basis for this (in fact, it's probably because of the original count that had me settle on a min and max of 10 and 100). Maybe a better question is: What are factors that studies consider, besides the logistics you rightly pointed out, in determining a sufficient N?

I answered it in another response, but I think this is why we're seeing a rise in meta-analysis papers. Take a bunch of small N's, consolidate them, and analyze their trend. This analysis can also be strengthened by evaluating the effect size of the phenomena [1]. However, I would say using effect in meta-analysis is a very complex approach that limits the set of researchers that could conduct the analysis appropriately.

[1]https://en.wikipedia.org/wiki/Effect_size

>I'm curious as to what makes a good size of N.

That depends on the size of the population being sampled from, the margin of error, and the confidence level.

For a huge effect like the one shown in the study, where one side performed 2x as well as the other, a sample size of 48 is more than large enough to say that the result is statistically significant. If there was as small effect, that wouldn't be the case.

Put it this way. You want to find whether people from California prefer Taco Bell or Pizza Hut, so you randomly sample 100 people. If all 100 people say Taco Bell, then you can be reasonably confident that more people from California prefer Taco Bell. Because if at least 51% of your population preferred Pizza Hut, the odds of not getting one of those people in your sample are minuscule (the odds of getting all Taco Bell people in your sample if 49% of the population prefers Taco Bell is 0.49^100).

If 51 prefer Taco Bell and 49 prefer Pizza Hut, your confidence level is too low to be useful--you need a larger sample size.

Of course I understand why the research has N=48, my question is whether it is adequate to fully support the claim in the title. I really don't know. I'm not a scientist or sociologist. My intuition tells me that it seems small, and using a population of college students intuitively feels like it wouldn't adequately capture the diversity of professional software developers, but maybe it does in this case.

It is; however, my next understanding is this is why we are seeing a rise in meta-analysis papers. It is difficult to acquire a sufficient N with a single study. As a result, you see authors consolidating all of the papers within a topic and discussing the trends seen in each.

However, research also has issues with reproducibility. No one will publish work that was simply a repeat of someone else work. Researchers have to twist or add something. It is a discussed issue, but no one in power has made attempts to rectify it.

> Also note, what is a "large N"?

Depends on the size of the effect you wish to measure, and the underlying variance. They got a massive effect, which to me, seems improbable, and I half suspect the randomization is at play.

I think it'd be pretty simple and cheap to replicate the things we care about in this study. All you really need is a whiteboard, some markers, a small room, a few interviewers and a candidate pool. The eyetracking getup really only gets in the way of that and seemed more like a problem in search of a solution.

> but isn't N=48 a little small?

Not when the effect is large. 48 people is fine when there is a 2x difference.

The point of a whiteboard interview for me isn't about accuracy and efficiency. It's about seeing whether the person can code and can talk to me about coding.

Everyone says this but has anyone been hired if they can’t come up with working code in the session?

Yes, of course. I rely on compilers, linters, and editor syntax highlighting when writing code, so it'd be pretty hypocritical for me to penalize people who can't access those things. People can sketch out a pretty messy pseudo-code implementation if they want, and then we'll just spend the rest of the interview cleaning it up, making it correct, extending it, talking about quirks of their implementation.

I was an undergraduate at NC State. I wonder how the study could be replicated at UNC and Duke. I'd wager they'd have a significantly higher success rate.

A company flew me out to San Francisco to interview once. On my way into the building in the morning I was mugged (almost right outside). I told my interviewer, they took me to the lunch room to sit down and compose myself. "Take as long as you like", they said. 2 minutes later they came to collect me for my interview.

I was a wreck. Worst interview of my life. Didn't get the job. I am pretty sure I would have if I wasn't falling apart.

Oh my god. I'm so sorry... that had to have been very, very traumatizing.

It sounds like a net win that you didn't get that job, for multiple reasons. I wouldn't want to work somewhere that I'd gotten mugged on the way in for an interview, nor for a company that didn't give me as much time as I needed to collect myself.

The closest I've gotten to that, was that I had an interview set up with a company in Saginaw, MI, that I was really excited about. I did a quick internet search to learn more about their company, and the first thing that popped up was that a few months prior, an employee had been shot in a drive-by in their parking lot.

I ended up cancelling the interview.

wow that is incredible, they didn't offer to postpone or anything?

That is seriously fucked up. Who goes to the trouble to suggest you take your time and then takes it away? That sounds like something a bully would do.

If they behave like that outside of work, I wonder how many breakups they've been through. Few people would put up with that kind of behavior and those people are probably wrong.

>All of the women who took the public interview failed, while all of the women who took the private interview passed.

Companies that are serious about avoiding gender discrimination in their hiring practices need to carefully consider what they are really looking for when certain candidates seem to have "the right stuff" after a whiteboard interview. We need more research in this area.

Given that women seem to be around twice as likely to suffer from anxiety as men, this is a result I was not at all surprised to see.

- https://www.webmd.com/women/news/20160606/women-more-prone-t...

Even if the science on this isn't settled yet, it's such low hanging fruit that I can't think of any reason why the big N companies aren't already doing it. Besides, it would require less time from the interviewers (since they don't have to be present for the reading and thinking portion).

The only good way to assess a candidate is to hire them. It's sad, but true. Some of the brightest and best will not even be able to code with someone else even paying the remotest attention to them. Interviews used to take account of this 15 years ago. Google came along and destroyed this.

I remember talking to a Google interviewer and I came up with a better solution than they expected. Since my solution was non-standard for online median finding-they wanted a unbalanced binary tree where I wanted a tree with the values at the leaves and using a fixed height, in this manner you could easily find successor and predecessor and I'd have actually found a better solution than their expected solution. They wanted the standard binary tree solution with predecessor and insertion, where I failed.

Everything will still jovial, until I mentioned that I consider the tools I use and how to operate them as part of my skill-set. Apparently this is false.

This was me interviewing at Google for a potential job using computer graphics or computer vision. I wouldn't even be working with problems anywhere close to this.

I've found that everyone is just copying Google's process because of all of Google's cash and success, but don't realize that most of Google's success is not due to any investment but rather just one solid team doing search well.

Anyways, I'm glad I didn't get hired, I have a better job (pay and work). Both of the projects I would have worked on have already failed or nearing failure already, it's been like 3 months.

I used to interview at Google and think you're misunderstanding why you failed the interview.

If the solution works and is better, then the interviewer should recognize that. I have this happen often when I use a new-to-me question (well, not often better, but similar enough performance). Even if the interviewer failed you for something they shouldn't have, it'd raise a flag in the hiring committee if you passed the other 3-4 interview sessions and the committee should look closer at the one failing interview to see what's going on.

And I have no idea what you mean by the "tools" and "how to operate them" being false and failing.

Do you understand why I failed the interview? That's absurd. How can you understand if I've not even given all the details? Your process is somehow bullet-proof and guaranteed to be followed? Probably because that's the assumption Google is predicated on. It couldn't have happened, so it didn't. I mean, logically, that's true.

> And I have no idea what you mean by the "tools" and "how to operate them" being false and failing.

I mean I consider my skill-set (what they're buying from me) to be my theoretical / intellectual ability, in addition to the practical activities I take when using the computer to achieve my job (using a mouse, keyboard, git, c++, testing). The interviewer mentioned they are looking for generalist programmers who don't consider their tool-set (git, windows, whatever) to be part of their skill-set. While I am quite general, I consider it a plus that I know such a wide range of different technologies. It was my question which prompted this, I asked about this very thing since I know it's a sore spot at Google, knowing a few Google employees and some Google coops. I believe this is the reason I failed, the interview completely changed his tone at this point.

In any case, the interview didn't test anything I would have done on the job, or my ability to do such things. It also didn't even test my intelligence independently of my preparedness, since there's so much preparatory material available, though it did eliminate people with lower intelligence who didn't prepare.

I failed the interview because of a number of reasons, but I definitely solved the problem. The typical process wasn't followed as this was right as COVID hit. So maybe that had some effect as well, this was the second phone interview with a new person (the first one I completed the problem, but there was a glitch and the call dropped and we couldn't recover for some time), and I've been ignoring the recruiter for years. I think I didn't really want to work for Google in the first place, so that probably also came through. I'm a big proponent of privacy and against Google's advertising business model.

I think based on the interview process I'm no longer considering Google anymore, despite still having another interview in six months. I will probably inform the Google recruiter in the future. It's just a waste of my time to prepare for such a silly test. When the main book to get the test is a book about how to cheat it and appear better than you are, it shows that the test is completely broken. Maybe a company the size of Google should hire people for short internships where they code throw-away prototypical problems and do some paid work. Maybe on a problem that takes some time and planning. I'd do that, even today. I think Google has lost most of the individuals who actually have an impact in this industry beyond search.

Maybe that's why the process won't change, Google isn't selecting for some traits which are commonly sought after.

As a hiring manager the test that I believe works best is a code reading exercise. This helps reduce anxiety or the effects of anxiety (fear has a very big negative effect on creative thinking)

I take a small fairly self contained function from the existing code base. Give them 10 minutes to read the function.

You can also add a few actual bugs into the function if you like to see if they are picked up.

I then generally ask them a couple of questions. First to step through what the function is doing. Then what the larger context of the function might be and finally if they have any thoughts on improving the function.

It's a quick test usually under 20 minutes you can even hand it over to non technical interviewers if you describe what kind of answers they should be looking for.

Interviews are a joke and are as useless as randomly selecting people.

I did 14 interviews at Google (two applications, first one after being contacted by them, for second one I contacted them to try again). As far as I could tell, I passed most of them, and for some of them I even provided new, unique solutions that the interviewers were amazed by. That being said, both times I got reject in the last step before the offer without being given any reason why. The only think they told me was that I should learn less from programming books and have more experience actually building things. I have never read a programming book, only learned by doing and had many projects to showcase. Some interviewers were not even paying attention to what I was saying, seemingly working on something else on their laptops. My recruiter was both times so certain that I would get the job (based on the very positive feedback she received from some of the interviewers) that she started telling me about the relocation process.

Today, 4 years later, I am not mad about it as I got to work in a very cool gaming startup for 2 years and now I run own company where I have created a product used by thousands of webmasters. I am honestly just confused about the process, as I know that I am a top-tier programmer and, based on what others said, a nice person to work with. I guess that ultimately it was their loss, but it's just a machine so no one really cares if X gets the job or not.

Some states have GDPR-like laws that will absolutely allow you to request a copy of your assessment. That can be a good way to know exactly what went short (most companies don't expect this and will get VERY coy about what they wrote, but the law's the law~)

Either way, more interesting is your first line:

> Interviews are a joke and are as useless as randomly selecting people.

I don't have a source, but I read a study once that showed most engineering interview process is roughly as accurate in assessing success as flipping a coin. I didn't look into the methodology, but based on experience I'm willing to believe it.

Other fields don't have that issue because they don't use interviews as a proxy for performance. Either they use portfolios (eg: designers) where you can CLEARLY see the person's skills, either they use credentials (certifications, or just resume). The interview is just to assess those data points and to make sure expectations are clear.

In some fields (eg: trades like carpentry), people will look at the person's history and hire them. Then based on actual performance over the first few months, they may get let go or even demoted. Can you imagine a big tech company DEMOTING a software engineer? Blasphemy! But really, that's the only way to properly assess an employee.

Hire them based on their experience and claimed skills, have clear expectations, adjust after a few months. Then you can give a shot to the person with low experience who claims they "promise to work very very hard and learn quickly". You can have apprenticeships too! And we can save everyone the trouble of silly useless interviews.

Multiple discussions about this on HN today, that's good. Maybe it will lead to the pendulum swinging back a bit.

Interviewers need to do their due diligence. Coding tests can have their place when interviewing for certain kinds of positions, especially for new grads and entry level positions. In the end nothing can replace checking references and looking into a person's work experience.

There is no algorithms and data structure whiteboarding test that can differentiate between a junior and a senior developer. Being senior is a product of acquired wisdom, not just skill. And in fact senior devs can easily bomb out a whiteboarding test because we're further from university CS experience.

Your last paragraph sums what why everyone on HN is so upset about the interview process. That is, the process is tailored for new grads, and what is being tested is not at all related to the job. 5 years of schooling, and 10 years of experience is not enough if you can't implement depth first search using double linked lists, hashes - and ultimately graph theory - within 45 mins. Heck, new grads can't even do so.

Well, yes, if you looked at my comments you'd see I'm part of that "everybody" set.

But to moderate it a bit; in my interview training at Google it was clear: the interviewee doesn't have to get the answer correct. The point is to see how they think.

I mean, I think it's still insulting, but there is that.

Yes, true regarding the very last part.

My first "real" CS-related interview, which had technical questions, I had on the very same day that I had taken my final exam in our data structures and algorithms class. And because our DSA class was quite rigorous, I knew the material in and out.

I absolutely aced those questions, but did poorly on the systems design questions because, well, I simply did not have much experience designing or building complete systems/software.

Years later, it was the other way. I stumbled my way through the DSA questions, while acing the systems questions.

But with that said, I think also the order of your performance matters. If you walk in, and absolutely bomb the first topics, I think there's going to be a bias against you. Maybe they'll think you're a moron, or something like that.

Bad interviews make for bad hires. Bad hires can cost a company more than not hiring at all.

Whenever I've conducted interviews in the past, I've gone out of my way to help the candidates relax. Need to use Google? Go ahead, I do literally all the time. Need some advice, something explained? Just ask, that's why we have peers. Nobody writes code on a whiteboard, with no docs, with three people watching, silently taking notes.

Interviews should reflect the environment within which the job is actually done, and allow the candidate to demonstrate what they're good at, instead of treating them like a circus animal jumping through hoops, giving them performance anxiety.

For my most recent job, they circumvented my interview entirely and just did a few days of contract work instead. That way the company gets usable work, the candidate gets paid, and it filters out those who can interview really well but may not be a good company fit.

>Bad interviews make for bad hires. Bad hires can cost a company more than not hiring at all.

But is that really true? There are plenty of companies who take a Fire Better approach (Amazon, Netflix to name a few) who seem to be doing just fine.

Not being able to tell if it's a bad hire is the core problem for the rest.

Worth noting that, at least as far as I've heard, the pressure on employees in at least some roles at the two companies you mentioned is incredibly stressful. I would rather have an interview without undue stress than a stressful one, but I'd rather have a stressful interview than continuous stress on the job. (I am differentiating here between "working on difficult problems" stress and "always wondering if you're going to get canned" stress.)

I hire for industrial IT, and being able to handle the anxiety of a high-pressure situation with grace (e.g. mill down, losing tens of thousands of dollars a minute) is an important factor. No one expects miracles, but you need to be able to keep your cool and work the problem, not waste time fighting with your own rising anxiety.

Seeing how they deal with a stressful situation is an integral part of the interview in my opinion, and I imagine it is a feature for any IT role where that is the case. Standard dev roles, or other roles where time-based stresses aren't as prevalent, may be different.

Only yesterday I had a candidate bow out 10 minutes into the whiteboarding exercise, which is probably best for both the individual and the company if that kind of situation is untenable for them.

This is certainly the most common rationalization.

But the reality is that the situations are very different and elicit very different responses.

I do great in a real production fire and stay cool and collected. Put me in a room with a stranger who’s judging my every move and my mind just goes blank

Stress when interviewing is similar, but not the same as on the job stress.

One major factor, with a job: you're still getting paid. I think the economic anxiety one faces is pretty stark, if you fail a job interview you're going to be unemployed until you can pass the next one.

Yes, interview stress is completely different. You have to commit a pretty egregious offense on the job to get fired on the spot, but literally anything you do during a job interview can cause you to lose the job.

Also, performing in front of strangers is completely different from performing in front of people you already know (co-workers).

It's extremely common for people to be nervous when meeting strangers but fine among associates. This is such a blatantly obvious empirical fact, it mystifies me how the tech industry can act as if this phenomenon does not or should not exist.

And often you won't have any idea what caused you to lose the job in a particular interview. There's generally no feedback at all so it can be very difficult to try to improve.

Yeah, devs that work for me also work in a critical environment such as yours.

As soon as an issue is raised by our clients, they immediately kick off a 40 minute long timer and put a physical whiteboard in front of us asking us to solve the whiteboard problem in front of them in under an hour. All the while the client is physically behind us commenting on our code and watching our every move as we paint the whiteboard with a solution. This is literally what happens to me and my engineers on the Job. All of our code happens on the whiteboard (we don't use editors or keyboard or mice!.)

The only thing we need to deliver to our client is a picture of our code on the whiteboard along with some trivial complexity analysis and our client pays us money for coding under pressure.

Our interviews just consists of me doing the exact same thing as what happens on the job and therefore our interviews are 100% identical to our real life coding problems.

When science tells someone that the interview process they've been conducting for years is essentially wrong people have to sort of remake reality and their own reasoning to justify the science rather than admit fault. Obviously, this isn't what's happening with me or you in this case.... the interview process at my company is 100% identical to real world scenarios so I'm good.

I've always been telling everyone I know since before this article that I interview for anxiety not useless technical skills; anxiety is what determines whether a developer is great or not.

I can’t tell if this is an actual white boarding purgatory you live in or a joke ?

Our company consults in a variety of software related things. For example in our interviews we ask our candidates why are manhole covers round?

We ask them this because, literally on the job our employees are suppose to program robots that make manhole covers. Daily problems we face include deciding whether we should program the robots to make square manhole covers or round manhole covers. Really hard stuff... and you really need to be able to figure out why manhole covers are round or how many golf balls can fit in a bus in order to do the job... If you can't figure this stuff out you're not qualified to work for my company.

Not to mention, as I said before, none of our programming happens using computers. We do it all on a whiteboard in under an hour while the client is over our shoulder watching.

It's fine if you believe this is true for your environment, but that's really quite rare.

I can see 'high pressure' particularly in Devops, but that's a different kind of problem solving.

The headline speaks perfectly to my intuition after many years in the trade, and that we are 'measuring confidence'.

Note that they assumed that interviews were set up to be fair - they are not - and that's actually ok. If a company has preferred candidates, it's perfectly within their right to do that (so long as 'preferred' doesn't just mean 'gender' or whatever).

It's been a while since doing this for me, but when I do it again, the 'on site' will be a page with some neat problems, some Q&A and then they take basically as long as they want to go through them, the interviews start when they are ready and you just go over 'what they did' and then interview for the rest of the issues.

'Having someone watch you solve a problem' is very specific social condition.

Honestly if you built a better bridge more would come over.

Looking at white boarding comfort and judging that the person can't handle stressful situations when they have a defined role and understand the stack are two different things.

I feel like everyone wants to make a quick judgement and pat themselves on the back.

People can learn quickly but no one believes it.

This largely applies to people that have already produced working software in the real world, but...

Companies should trust the people applying based on their resume and ability to talk about what they have done in the past or how they would approach real world problems (organizational, interpersonal, technical). Maybe have them write some small bit of real world code using libraries and frameworks they are familiar with, while they are alone without anyone watching [key|pen]strokes.

Maybe I'm lucky but after 20 years in the industry shipping products to consumers I've never ran into a Software Developer or "Engineer" in the real world that couldn't get things done in a normal working environment in private industry (now in government, don't get me started). Some are slower, some are faster, some can handle bigger problems, but I have no idea how to reliably filter for that in an interview having tried many things.

The "OMG this senior person can't write FizzBuzz on a whiteboard or while videoconferencing and knowing someone is watching your every keystroke so they must be a horrible programmer" approach is just wrong.

I've only ever failed interviews where I had to solve problems or write code with someone watching (virtually or in person). Let me be alone and give me a product to build, though, and it'll get done. Customers will love it and it will be great. See: resume and products in the wild. The Math-oriented abstract tech interview is missing out on all the practitioners that care about the product and can build reliable maintainable systems in a team environment.

These days I'm primarily management, though, so I get to set the hiring guidelines. Yay!

Could you imagine working with a senior software engineer that couldn't write FizzBuzz? Anyway, I feel your approach may end up selecting for prestige and be more venerable to implicit bias. A recent engineering graduate from Cornell will probably have a much nicer looking resume than someone from some random state university even if they both have strong technical skills.

No but I can imagine a senior software engineer that couldn't write FizzBuzz during an interview. I would probably stumble on it myself. I prefer to look at what people have done and not their credentials. I don't hire people that haven't shipped something.

To clarify, by a senior engineer I don't mean a manager, I meant someone that is going to contribute technically in a meaningful way to a project (by writing code). I feel like it's a big red sign if the candidate doesn't know how to write a for loop. People will lie about experiment when they are desperate for work. Moreover, there are some people that are just great at talking...

This is why I stopped writing code. After I bombed a bunch of interviews I threw up my hands in disgust, it wasn't me it was the process.

One thing that stands out to me about the process is that everyone gets the same interview, doesn't matter if you are fresh out of school or have a lifetime of experience. Experience counts for nothing in these interviews. Can you design twitter in 40 minutes, thinking through every possible failure mode and issue along the way, or not? Apparently, I can't or not well enough. I think these companies really want folks with great public speaking and whiteboard skills - I'm kind of an introvert and that is a challenge for me where the day-to-day work is not.

"This is why I stopped writing code."

would be interesting to hear about what career you transitioned to, if you're willing to share.

I have a background in IT. I basically moved back to IT. I still write some code but its not the majority of my work. The code I write now relates to automation and integration and not so much building a product from the ground up. I have to build for reliability but I don't need to build for Internet scale.

Edit: I should say that the interviewing experience that put my over the edge was actually not a whiteboard but a take home exercise. What I didn't know and didn't appreciate was that it was designed with an adversarial review. There was a long list of requirements and I thought I met them all. The response was kind of rude and was a big turn off. After a second pass I was told that I was missing some fundamental things, which were never requested or discussed. The whole thing seemed designed to exclude rather than assess.

So you gave up on everything because of 1 experience ?

No that was the last in a long line of uncomfortable interviews. It wasn't worth the effort to me to become a walking algorithm encyclopedia, most of which I'm never going to use. Why would I continue with a process that appears designed to exclude me?

Honestly, its their loss. I've done great work and continue to do so. For better or worse, may name is on patents from more than one job and from different industries. My work experience and history of accomplishments says something about the kind of candidate I am. Hounding me at the whiteboard or calling my take-home work crap seems unnecessary - if that is what the job is like then I don't want it.

And I didn't 'give up everything'. I make the same money in IT without the hamster wheel feel of always 'sprinting'. Marathon runners do not sprint the entire race.

What is "IT" ? Do you mean sys admin ? Is that why you don't want to learn algorithms ?

Here is the problem, many people have patents, many people have words. You should be able to solve a simple BFS, DFS or DP problem. These algorithms ARE used in good jobs. Seems like most people here write CRUD apps.

Also, lots of FAANG engineers have lots of patents as well. That doesn't mean anything.

I certainly know my clinical anxiety has caused far too many interviews to crash and burn spectacularly. It'll be like "wow, you're graduate educated, 16 years experience, you must be hot shit", and then I go up and just flounder. For me it's not so much writing on a whiteboard, it's having someone standing next to me, or sitting in a chair looking up at me, occasionally interrupting me. I'll try to think out loud and all of that stuff, but that seems to invite questions that sound judgmental to me. I'll start thinking "they don't believe in me, they don't like me, they think I'm a fraud". Unfortunately, I've had those fears confirmed some times too. I had an interview end with a guy saying, as he handed me off to the next person, "You really are the lead of the ... department?"

I experienced this personally as I bombed a whiteboard interview because I'd been studying scala to work with them and programming a side project in C, and literally blanked on simple ruby syntax and idioms when I started to panic. When we started our own consultancy made 2 simple rules:

* Give people a take-home test that mimics the real work we do.

* Pay them to do it.

It's amazing how many candidates we find that are great at the actual work we need to do, and happy to do it, and how many people it weeds out who wouldn't be happy doing our normal work.

Honestly this seems like the best way to do it

My 2c. This is purely anecdotal, and based only on hiring about a dozen developers.

The best way to hire good developers is to take them out for coffee/lunch/beer and talk shop for an hour or two. That's it. Make it more of a discussion than an interview. Ask them about the their experience, what have they liked working on, what have they hated, but also tell them about what you've been working on and see what feedback/ideas/questions they have about it. I find fielding their questions and comments just as valuable as asking them my own. Joke about things that have gone spectacularly wrong, trade war stories, that kind of thing.

I think it's a great way to assess whether a dev "knows their stuff" or not. A couple of other notes: - If you're not a dev yourself, get someone who is and you trust to do it. Try and keep it "dev only" if possible, remember this is supposed to assess their dev ability, but is also a good indicator of communication and cultural fit within the dev team. - Try to do this one-on-one or at most 2-1, any more than that and it becomes a panel interview. - If someone is regurgitating theoretical or blog-post-scanned information without much understanding, it becomes more apparent the longer you chat with them. So an hour or two is a good length of time.

We usually also do a quick take home coding test too, but have found the chat/discussion to be better indicator of long term success. As many have already pointed out, I also find the coding tests (whether take home or in person) don't really assess "real world" ability and can often be gamed.

I can relate to this. There are two interviews that gave me really bad anxiety, but I think it's because I went in with anxiety as both required moving across the country in different directions. The first I was accepted, but I think only because the hiring manager worked with me before. I felt short of breath and like I wasn't listening to half the questions. The other was Facebook, which was probably the hardest interview I'd ever done. One particular question involved a lot of keeping things in your head and math (network/node design), and I just froze and got frustrated and in the end gave a crap answer with no thought. Given a half hour without someone staring at me waiting on an answer, I probably could have came up with something decent, I figure.

I'm not particularly upset at either case, I'm happily employed. But there was a nagging feeling in the back of my head that I was disappointed that I didn't get to convey 'the true me' to either company, whatever that means.

I went through the interview gauntlet last year. When my extended family would ask what it is like I would say:

Imagine you want to hire a comedy writer for a TV show. However, you conduct your interview by having your candidate perform stand-up improv comedy like "Who's Line is it Anyway". While you want to hire someone funny, you don't need to hire a performer.

In my experience, programming interviews are basically an improv programming performance. I didn't pass an interview until I had enough practice performing while others watched.

I agree with this. It really is putting on a show.

Ideally you get a problem that you are familiar with. Even more so if you know the exact problem and optimal solution. If not exactly, then at least similar to something you've solved before.

Then you put on a show acting as if this is the first time you're seeing the problem. With practiced confidence, you roughly outline a naive solution, then quickly and smoothly iterate your way to the ideal optimal solution. All the while explaining your "thought processes" as if you serendipitously saw the solution materialize before your eyes.

By practicing (say, grinding leetcode), you ensure that you have the confidence to smoothly put on this show, and also not waste your precious interview minutes meandering down suboptimal dead-end paths.

>. For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed

Talk about buying the lede. If there's any justification for abolishing this practice, that's it right there.

I agree that's a really striking and suggestive result, but keep in mind the sub-sample size there is four passing on the private side and six failing on the public side. My guess is there's a real and meaningful effect there, but it may not be to the degree that sentence without context would suggest.

Having performed hundreds of tech interviews, I realized that my developed super-skill was to keep the candidates relaxed regardless of how well or bad they did.

That meant carefully controlling my emotions when their answers were bad, and having a smooth transition when the answer didn't come at all - a situation that typically causes tension. You have to essentially hand out the answer or transform the question into a trivial one so that the candidate can have a small win and relax. If you're not doing that, chances are you're screwing up the whole process by throwing the candidate off-rails after a simple non/bad answer.

That's why the interviewer has more control over the outcome (besides the scoring) as his / her bias will heavily reflect on the dynamic.

That is indeed a super-skill! If you remove as much of the perceived power imbalance between interviewer and interviewee (often difficult, to be sure), and you can engage in a two-way conversation with them about the experiences on their resume, then you'll get a far better picture of how they'll be as a team member.

A candidate's ability to do performative whiteboarding while 1-5 interviewers stare at them tells me very little. For a software engineering job, of course technical ability is important. But equally (and often more) important is the amount of friction they could potentially introduce to a well-functioning team. What I want to know more than anything is: "Can I stand working with this person?" "Is this person a dickhead code-reviewer?" "Will this person be able to adapt and overcome when presented with a novel or urgent task?"

The way to divine these answers is by trying to put the candidate at ease as much as possible. So many interviews, unfortunately, become a sort of gauntlet or inquisition.

This is why internships are so important. Sometimes it’s still hard to tell after 10 weeks. It’s very naive to think we will know after a 30 minute interview.

I don't think people are naive, they just don't have a better solution. Most established professionals are not interested in a 10 week trial period after which they might be back on the job market.

"Most established professionals are not interested in a 10 week trial period after which they might be back on the job market."

Why not? If the position isn't a fit for either side after X time on-the-job, that's OK right?

Because maybe they support families with their job? Unless they had a ton of savings, parents aren't going to switch jobs with a chance of being unemployed again in 2.5 months. I wouldn't do it and I'm young and single.

But isn't the chance implicitly always there?

> Why not? If the position isn't a fit for either side after X time on-the-job, that's OK right?

In the US at least, health insurance is a big one.

How are you going to convince me to leave my current job for a 10 week trial period? That right there loses you the set of all people who are currently employed and are fine where they are right now.

There are a few issues with the scenario you described.

For health and safety, security and tax purposes you would likely not be able to have an extended period of more than a couple of days without some form of contract for that 'trial' period.

If you are an 'established' professional then you most likely would interview for a new job while currently employed. So even in the remote scenario that you would take a couple weeks holiday for a 'trial' you would likely violate the contract with current employer by signing a 'trial' contract.

This would only work for junior positions or for senior professionals who have taken some time away from work. But if you make this a part of your recruitment process, there is the risk that you are discriminating against 'currently employed' candidates.

Some places have a contract-to-hire approach which is effectively a trial period.

I believe companies fall into two basic structures:

1. agency / consultant model - implement short-term projects, specific tasks, etc. It's easier in this approach to swap employees in and out as new projects start.

2. software product model - the company's entire business model is built around or highly dependent on internal software projects. Institutional knowledge is very important. It's really important to protect the product process and plan for the long-term implementation of strategic vision.

Contract-to-hire works well for #1. But for #2, the overhead and cost of disruption to the core process is too high. Top-tier software products require a relatively smaller number of highly engaged and skilled people to drive projects to an optimal point. These people need to also be directly involved in hiring and onboarding to maintain the culture and high standards. Bad hires can destroy such a team. Thus you end up with the current interview processes as they're efficient and err on the side of rejecting qualified candidates accidentally. If they spent 10x as long on hiring, they could improve the accuracy, but then they wont have as much time to build product which is the key tradeoff involved.

Other people have mentioned the logistical/financial problems this incurs, but one other important consideration is that you are competing against other companies that don't do the trial-to-hire process and can thus offer more stability. And to be honest, given the state of the software engineer market (in tech-heavy cities), you already get to spend a couple of months trying out a company and then leave if you want. The difference is that the employer isn't similarly able to get rid of you after 3 months (it's too short a time to evaluate a new employee, the company has already invested heavily in onboarding costs, it's much slower to backfill a position than it is for a software engineer to find a new job, etc).

I would wager the opposite. That many people looking for a job are open to a (paid) trial week, when the alternative is countless grueling interviews that lead nowhere.

It depends on who we're talking about, but for high quality candidates in software heavy cities, this isn't very true. They are getting tons of offers and there's no reason to accept a trial offer. Plus you can't trial until you've quit your current job so for many people the whole premise makes no sense. It's not like you will have to interview with fewer places, it's just that each interview takes a lot longer.

Paid trials also have the issue that they might interfere with unemployment earnings while also being too short to earn unemployment afterwards. So they probably don't work for people with a job and don't work for people without a job.

Yep, that's the only intractable problem, so trials either go unpaid or pay without filling taxes, which is somewhat shady.

It's a myth that developers have a ton of offers. There are few developers who got lucky to have multiple offers at some point in their life. It won't last forever. They will eventually face a bad situation trying to get a job after being laid off or being on extended paternity or medical break.

Consider the current recession with record high unemployment, countless employees are desperate to get a job, including developers. Yet companies still prefer to have impassable interviews and hire nobody, than to give a trial.

I don't think this is true because taking a trial week requires leaving your existing job for... a trial. Why would you do that?

I don't see how internships help. I get the point that at the end you know if they're a hire but you still had to filter who to intern in the first place so you're still back to interview people to be interns which is the same problem you had.

Also, training a new employee or intern in your systems, code, company procedures etc takes a bunch of time and investment so it's not like you can intern some higher multiple of people than you'd hire.

I think it's effective for college candidates. You pay them much less, they often work on greenfield projects where learning existing code/process is less important, and the company gets to try out more candidates than they end up hiring. All of that while still doing right by the candidates.

The distinction for interns is that there is a definite end to the internship, no harm no foul. An employer can easily pass on every intern that is not a perfect fit without tarnishing the interns' reputations. That allows a less restrictive filter to be applied in selecting interns.

Let’s say an internship increases the chance of your getting a good hire from 50% to 75%. In that case 10 weeks will pay for itself.

You say that as if your boss can't wake up one morning and fire you because they don't like the way you pour your coffee. The only thing that changes is how much paperwork they have to do to make it happen.

This in no way jibes with my experience as an employee. I know the value I provide, how long it would take to replace me and how much the company has invested in me and there is zero chance that I get fired tomorrow. Theoretically my boss could wake up and fire me, but practically there are many reasons whey they "can't". Similarly, it is very rare to see someone get fired in the first 3-6 months on the job unless they are ridiculously unprofessional or unqualified. There are high-profile companies where it could happen, but they are noteworthy because it is so out-of-the-ordinary.

They can't in general, it's against the law to do that. And there are penalties if caught (and being caught is not so hard.)

Very dependent if they have a job already. If they have a job already, references from people inside the company are a very strong signal. If they’re currently unemployed (or a student) the internship is very valuable.

The next challenge is it’s unfair to weight too much on referrals because it punishes people who don’t know insiders. It also fuels a monoculture.

True, but who with options is going to agree to an internship?

Very good point. If you have options, then your work reputation will have to suffice. (Also imperfect because it excludes)

Mmm. Most interactive techniques that take longer than an hour or two seem completely useless as soon as someone has options. It's odd to me that all of this stuff seems so tailored to graduates, but nobody seems to adapt it as you go up the seniority chain. I feel like I'm missing something.

I downvoted you for two reasons.

First, expecting someone to commit to only working for a short time period is discriminatory. Only certain people could commit to such a thing, such as those with a low cost of living (no family) or those who are already independently wealthy. No regular professional with obligations is going to commit to that.

Second, if you can't tell whether or not a person is a good employee after 10 weeks then you're not a good manager.

One of the best hires I had made had a car crash interview. They could hardly speak they were so nervous. I could see from their CV that they had some good personal projects and thought lets give them a chance, that's what prohbation periods are for. On the job they were amazing. It was great to see them grown in confidence and they were a key figure at the company after a couple of years.

I want to push back on this a bit.

Writing working code well and fast is important.

But so is communicating with your coworkers!

Being able to talk about your code is not a weird thing you only do in interviews. It's an essential part of your everyday work, and the programmers who don't do this well are in a sense underperforming.

So just removing the talking part, as this study did, removes an important part of the interviewing. When I interview people, I value their ability to talk about the problem and how they think about it more than coming up with great code quickly. I need people who can talk to me, not just the compiler.

That said, talking through your whiteboard programming task might not be the ideal way to measure this skill either.

Our industry has created this mess thanks to that stupid crackin “book” and process designed by someone who never actually worked as a software engineer or released and maintained an actual product.

When I “cracked” my Google interview and got the job, I joined a team I never met. Of course we all could code , since we “cracked it”, but did I enjoy the team dynamics or team culture? Nice people but if they were a 10 person startup I wouldn’t choose them or would they have been successful.

None of the social dynamics which are important to teams was there or even fostered, acknowledged.

You just did a transfer and kept fishing.

Why, b/c Google just believe coding skills are all that mattered.

Yet they matter, but so do other things.

One of the best interviews that I ever had was a phone interview where I just so happened to have a good session of meditation beforehand. I never realized how much how well I could do in an interview until I did one while feeling relaxed.

Did you get the job?

I got an in-person interview, which didn't go as smoothly.

Did you meditate before coming into it?

Yes but my skill in meditation is no match for 4 1-hour coding interviews back to back.

When I interview a candidate, I'm explicitly and intentionally looking at how they respond to stress, how (if) they're able to say "I don't know", and how they communicate complicated ideas. I open with soft-balls to put them at ease and get them opened up and talking. Then dive deeper into technical things. All the questions are technical, but the main answer I get is how they communicate, how they relate to people, and how they deal with the unknown.

If you understand that a technical interview assesses anxiety, and you roll that into the plan, if you know what the game is, then you're 1/2 way to a win-win already.

yeah, but these anti-patterns are completely different everywhere. your insight isn't a signal at all, when the next interviewer isn't saying what they mean and evaluating only correct answers.

I'm not commenting on how anyone else does it, or saying that there aren't bad patterns out there. I'm telling the room about a strategy I found that lets me twist the bad into good.

I understood that, I want to highlight to others how there needs to be standardization or a completely different approach, as this is all a crapshoot and wasting countless work hours.

Assessing anxiety is probably illegal since it’s a psychological diagnosis. Just saying, if that’s the goal that’s both messed up and legally questionable.

Interviewing is a skill. I once had a good job where I was happy and didn't move for 10 years. Big mistake - after that I bombed at interviews. After doing 10 interviews I'm now up to speed. I kinda feel like I need to do some interviews every year just to remind me how to behave and how to keep market ready.

I was in the same boat. Nearly 10 years experience at the same company that hired me after university. My first phone interview had me write a linked list and I froze. I had even practiced writing linked lists that week. The interviewer probably thought I lied about my resume.

I spent 4 months doing leetcode problems and reading books before I started doing interviews again. I would interview at places like Facebook that I had no interest in working at but I needed the experience. 7 months after I started I finally got an offer.

I don't want to go through that again but I agree that you probably need to do interviews every year inorder to not completely lose that skill.

Traditional tech interviews (live coding challenges, algorithms, brain teasers, etc) have so little value and cause so much suffering in candidates, they are borderline sadistic IMO and in the end you are just as likely to make a bad hire, then not be organizationally prepared to deal with that so you end up with a broken team and bad outcomes. Why?

There are still challenges with contract to hire but overall I've had great success with it. First of all, we get a realistic deep look at candidate capabilities before we commit and they get a look at what it's like to work with us. Then we're very prepared to part ways because that is the entire point of the contract to hire. If feels 10x more successful in hiring great people than any other approach I've personally seen and, very critically, it is more humane and respectful.

There is a hypothetical concern that highly pursued candidates won't wait around for that but we've found exactly the opposite is true. The best developers are happy to see we have a more sound hiring process and are psyched because we have a credible case that our teams are really high quality and free from the random bad apple that spoils the work experience at most other companies. Great devs aren't worried about being hired today or in two weeks, they know they can get a job anytime they want. What makes great devs great is usually their passion for great craft, process, culture - if we can credibly promise that, they overwhelmingly are happy to do contract to hire.

I failed the third phase of the Toptal interview process (where you have to solve two coding challenges, each within 20 minutes, while screen-sharing with a human observer). The event was nerve wracking.

On the first challenge, I only completing coding and did a first run of my solution with 90 seconds to spare. It didn't give the correct answer, and I had only about one minute to debug and try again. In that case, I felt immense anxiety and nearly gave up. But somehow I managed to keep it together long enough to spot a misplaced array index [ ] reference and produce a correct result.

On the second challenge, on my first run at about 17 minutes in I discovered that my entire approach was not general enough to meet all the possible cases. As I recall, while the rules of the problem are stated, you are not able to see test cases until you run. With three minutes left and the understanding that my approach was fundamentally flawed, I just gave up.

The experience was demoralizing. On the plus side, when I did live coding challenges for job interviews after that, I found them all comparatively easy and relaxed.

What would be better, in my opinion, is an overnight problem to solve, followed by a detailed walk-through of the solution the next day. That presented walk-through can prove to the interviewer that the candidate did not simply copy a solution (without understanding it) from elsewhere on the internet. Of course, if the solution was copied by understood, that should be positively accepted in many cases. There's a reason we use trusted libraries instead of writing our own code where possible.

I had a very similar experience with Toptal and going down the wrong avenue when solving the problem.

The interviewer muted themself and went non-responsive while I was working on the problem. So I was asking questions and heard nothing back at all until the time was coming to an end. At that point the interviewer came back to assess what I had done.

I remember it being a really terrible experience

This can be said of all interviews, not just tech. It’s hard for great many people to put themselves out there. I mean interview itself is a performance, isn’t it?

Having said that communication and interpersonal skills are big part of most jobs even in tech. We hardly develop software in isolation these days.

I don’t know about other industries, but in Tech there are some really inexperienced interviewers out there, who’re impulsive and lacks discipline in conducting objective and meaningful assessment.

I see lots of posts critical of companies and interviewers. The problem is there are too many qualified candidates for some jobs and companies must weed out candidates somehow. If they have 100 qualified candidates for 1 position, how would you rather they pick? The real problem is something else. It's a lack of jobs, inequality, the cost of living, and I don't know what else. Many people are getting squeezed.

It's not due to lack of jobs, at least not in a simple way.

Reportedly, there are simultaneously more open positions than companies can fill with qualified candidates, at the same time as many qualified candidates struggling to find open positions they can fill.

I think that may be true, even though it sounds paradoxical.

The result is still inequality, cost of living peoples and people getting squeezed.

But adding a bunch more of the same jobs won't fix it, if the problem isn't caused by lack of jobs.

I think the more accurate headline is likely to be "anxiety can negatively affect performance in interviews."

I would love to see what a control group of non-tech interviews looks like. Not sure what the equivalent "whiteboard while out of the room" is, but it's hardly just software devs who get nervous and can panic in interview situations.

There is a lot of anecdotes in the comments about these interview processes.

The ones with the best data are the company themselves, especially those who interview thousands of candidates a year. These companies (e.g., all FANG) have enough data to assess which interviewer, which interview questions and which interview process have positive correlation (and even causation) with future employee performance. And they can optimize the pipeline to interview most candidates with the interviewers that can actually assess future employee performance for instance.

Either these companies have already done these optimizations and analyze the trove of data they have, and the process is what it is for a reason. Or they haven't yet and there is still much room to optimize the process, and applicants have a chance to see those processes updated for the better (although here better means better for the company, not necessarily better for the applicant).

There is no data could collect that would tell them whether they are efficiently sorting good candidates from the herd.

Say that I wanted to sort men from women to only identify women. One thing I could do is have the candidate walk under a bar that was 5 foot 4.

Most of the people who would get through would be women. But this would not be an efficient screening method. I would still be losing many women to the screening method.

If I am only polling the women who get to the end of the screening to see if they are women, I can't evaluate the efficacy of the screening. Facebook/Google would need to hire some of the failed candidates to compare results.

There is literally no way right now for any company to assess the false negative rate, i.e. how many well-qualified engineers they turned away. They can only assess internally the false positive rate, i.e. how many engineers they thought were going to work out that did not. The fear of making a "bad" hire, is so pervasive that we optimize entirely to minimize the number of false positives, completely ignoring that making the interview more and more difficult is probably having a marginal effect on reducing the FPR but is drastically increasing the FNR.

"A new study from North Carolina State University and Microsoft finds that..."

Microsoft is studying this, because they realize that they don't in fact have "the best data".

Tech interviews are kind of like a lot of games in life: do something difficult and boring in order to survive, and make it look easy and enjoyable while you're at it.

This is bad news for URMs. I am hispanic and was an international student— looking back there's many "extra" things that added to my feeling anxious before interviews. The imposter syndrome was real and I worried about wether I'd fit in, for example. I luckily don't have much of a language barrier but there were definitely a cultural barriers.

In the back of my mind I was always thinking I had to do more than "enough" and be exceptional. You read studies about how two people with the same qualifications might not get the same results after an interview due to unconscious biases by the interviewer. I felt like I had to be good enough that'd I'd overcome any kind of bias the interviewer might have. It's stressful.

When I last did a serious job search a few years ago, I thought I was pretty hot stuff. Nonetheless, I spent a few months reviewing algorithm text books and practicing code interview problems.

Then my first phone interview came, and I totally bombed it. And the next one. And the one after that. All in all, I think I bombed about seven code interviews (didn't even make it to an onsite) before I started to get some traction. One of the interviewers outright told me that I wasn't a very good coder.

After that I managed to get a couple of decent offers, but the process was quite humbling. If I interviewed again today I'd probably be in the same boat. Whiteboard interviewing is definitely an acquired skill.

Which is why you get a lot of over-confident engineers who are good at memorizing data structure/algo questions and talking BS but are terrible at real day-to-day SE work. The modern tech interview passes over so many great potential engineers.

I have extreme performance anxiety when it comes to interviews. Recently I had a very good experience interviewing with Netflix, where the take-home assignment allowed me to show what I was capable of doing. All the technical interviews after that were talking about the take-home at a high-level. The experience felt more accurate to how actual work is (receive an issue ticket, do work, then a code-review).

I was rejected in the end, and I believe I either a) wasn't a culture fit or b) they found someone with more experience (I'm 6 years in at my first tech job). What was nice about it though was that I was very calm and collected throughout.

This is only true at FAANG, guys, stop working at FAANG its not doing the world very much good and there are tons of companies around looking to hire you that are. The future is starting now and FAANG engineers won't be a part of it.

In my experience, every company wants to be a FAANG-like company and so they've all picked up the same BS interviewing style.

Lol right. You're building the future on so many open source libs from FAANG.

I feel like jkingsbery's comment[1] is relevant here:

> One of the interviewer asked me to recite the algorithm for constructing a Convex Hull. > ... > we struggled together for an hour, with him frustrated that I couldn't remember the details of an algorithm I last had seen 6 years earlier and couldn't recite in 60 minutes what took our professor 180 minutes of lectures to cover, and me frustrated that would have taken me 30 seconds to look up.

[1] https://news.ycombinator.com/item?id=23847814

I just spent a huge amount of time interviewing for a new job. They put me through an absolute ringer — something like 10 total screens and interviews, over about a month.

At the end, I started feeling a really negative association with them. I would never sleep well the night before the inevitable next interview and it just got to become a really tough thing. And most of the interviews were totally no-nonsense, no pleasantries, just tough.

I ended up getting the job but I turned it down. The interview process told me everyone I needed to know about the company culture.

I had a similar experience. They wanted me to do 4 phone interviews before even bringing me on-site, stretched over a month. Don't really understand the rationale for dragging it out so long.

> rather than whether the candidate is competent at coding

As a really quick reaction it's worth noting that coding is probably less than 25% of my job as an engineer.

In others' experience is this also the case for you?

I just went through an job hunt. I've been in tech for nearly two decades, and tech hiring has gotten definitively worse. It was never good to begin with, but it has become more painful from a candidate's perspective.

Employers are so focused on the well shared "the worst hire is a bad hire!" that they implement obtuse and ineffective hoops candidates must jump through in attempt to weed out "bad" candidates. This results in hiring those who survive the process, not those that will do the best work.

It seems like loads of comments consider "someone good at coding" as the only benchmark of a good employee. Even the report implies that "people who took the test in private passed". Passed what?

I am looking for people who can handle pressure, who can communicate clearly, who can handle the unfamiliarness of the experience, who can handle time pressure even though most of the time they wouldn't face that at work. If an experienced engineer cannot split a string in less than a minute on a test then they don't figure as a valuable emplyee to me. Why? Because anyone can Google something or apply deep thought in a private room but good employees imho can work with noise around them, with distractions, with changing priorities etc. in other words the interview is a poor representation of a real environment.

Sure, it means you might lose out on someone who just needed a few hours to calm down but how are you supposed to know that? Maybe they needed another few hours, days, weeks? Maybe the test you gave them was hard but if you asked them about something else, they would have done better.

It's no different than someone turning up late. It very likely wasn't their fault but you use it as an indicator anyway because you don't know them and don't have much else to go on.

'but good employees imho can work with noise around them, with distractions, with changing priorities etc'

Yes, then they burn out and the cycle continues ...

There are literally dozens of reasons that tech interviews suck. However, they all mostly boil down to the same root cause: there's very little correlation between the content of most interviews and the jobs people will perform. Some real life situations that I've been on one side or the other of:

* Companies doing a "centralized" process that ignores the candidate. The questions are always overly generic and it really boils down to, "You need to want to work here enough to basically take any job we'll throw at you." Sorry, I don't want to work any company enough to just take any random job.

* Coders asked to complete academic problems that will NEVER come up for them in their real work.

* Coders not being allowed to leverage external resources like they will literally do EVERY DAY in their real jobs.

* Managers being asked to be coders in interviews.

* Questions that have binary "right" and "wrong" answers.

* Asking questions that have a million ways to do it and then honing in on one specific disagreement the interviewer and candidate had.

* Interviews that are more about the interviewer stroking their own ego and proving that they are smarter than the candidate.

* Completely ignoring that the absolute best indicator of future performance is past performance; not, "what quiz could this person solve in 45 minutes in an interview."

The corollary may be that anyone with low social anxiety is likely to do better.

https://en.wikipedia.org/wiki/Dr._Fox_effect

I'm genuinely curious - do whiteboarding and code exercises mititgate this at all?

Clearly there has be some minimal standard of ability. But given two similar performers, do performance-interviews favour those who project confidence? Or is there no influence on outcomes?

I had my first coding test for an interview recently (second dev job but a decade of other experience), over a video call, shared screen in a pre built AWS in browser IDE and I assumed I tanked it because it didn’t go perfectly. In hindsight it went like a typical 40 minutes of coding, I started down a path, didn’t quite work, had to console log to see what was up, tried a new tact and then it worked. They were clearly looking to make sure I had some basics of Angular down but also that I could problem solve in the moment and see where I went wrong.

I enjoy interviews (I like meeting people and chatting about their projects and problems) and I think running a VC backed company for nearly a decade has set me up well for detaching my performance in the interview to my self worth/abilities (ie. I’ve grown a pretty thick skin).

Ultimately it comes down to “can I do the job?” and “is their process set up to accurately reflect that?”. If the answer to the first is no then that’s cool, I’d probably have a miserable time in the role and under perform even to my actual level. If the answer to number two is no then I probably don’t want to work for them anyway as they are unable to accurately measure and analyse prospects/opportunities beyond just hiring.

I'm pretty proud of the interview loop we use at DefenseStorm:

- First section is "pair programming". We try to replicate what coding on the job is actually like: you can whatever IDE/REPL/language you want and can use google to look up docs. Our problem is specifically designed to be a fiddly problem which leads to edge-case handling with no possible 'beautiful algorithm' answer. Once they get the first solution, they have to revise their code to support a new fiddly use case. One interviewer actually provides help: pointing out syntax errors and suggesting alternate approaches if they appear to be heading towards a dead end. Most applicants "succeed" - so we evaluate on how well they can keep their code organized and readable, how they respond to suggestions, and if they are easy to work with/good communicators.

- Second section is code review. We give them several snippets of not-so-great code and ask what they would suggest we change before putting it into production. Does a great job of showing us where the candidate has depth of knowledge, and is a good approximation of a portion of the duties we expect them to perform. I was surprised to learn that very few companies do this - we find it extremely valuable.

- Third section is a conversation about tool familiarity (can you linux/aws?), depth of knowledge in the various domains we use regularly and designing an architecture (indicative of developer seniority). This is probably the least remarkable section.

Every section is set up so there are no gotchas and there are always multiple correct answers. I think we do better than most at minimizing the anxiety inherent to the interview process.

That is very close to how I interview people. The worst thing about white board coding is the adversarial relationship between you and the interviewer. It’s so much less stressful when you can team up with the interviewer to work on a problem together, and it’s much closer to being a real work sample test.

Coding interviews are not about coding skills, they are about communication skills.

I thought I had discovered a clever hack when I was interviewing. You ask enough closed questions about the problem that the interviewer tells you the answer. Then you repeat the answer back to them, as code or aloud, and you pass.

Now that I am interviewing, I am desperate to find anybody who will ask me enough questions to have me reveal the answer, and then repeat it back to me.

Like the “guess who?” game?

Used to run interviews by putting candidates at a table with a laptop (typically their own). I'd give them a coding question and 45 minutes to solve it, and then spend 10 minutes walking through their solution together. I'd sit across the table from them and do my own work. They could stop me and ask questions all they wanted, just like we were teammates.

It worked great. It also gave me (the interviewer) some time back in the day.

The only thing that interviewing assess is how good at interviewing the candidate is.

I swear that for 90% of the typical software interview loops, someone with some minimal understanding of computer science that spent 2 month studying the typical Leetcode problems would probably have been fine and do way better than a software engineer coming out with 10 years of experience that didn't take the time to review his leetcode problems.

What are people doing as alternatives to these kinds of interviews? The research is quite clear that technical interviews are worse than useless (literally -- they will select for the wrong traits in your candidate pool), yet they still seem to be the dominant hiring practice.

My current company does a few rounds of low-key chatting, some about technical stuff, then a 1 to 2 month trial period with the understanding that there is no guarantee of a job at the end, it only becomes permanent if it's a good fit on both ends. Obviously this is time consuming and expensive, but it works because we're a small and very stable company, which I realize is very much _not_ the norm. This does work great, though. They've managed to build a team of excellent, autonomous, highly skilled developers who stick around for the long haul. Wow, that sounds a bit self-aggrandizing -- I'm really thinking of my co-workers, I hardly count myself in that category. Maybe someday.

Anyway, I'm genuinely curious what other models you all use that work well. By "work" I guess I mean result in high quality and long lasting additions to a team.

I like that model your company is using. Though there is a major drawback: your sapping the momentum of your candidates in their job search. If they sign on for the 2 month trial and it doesn't work out, contacts in their job search will have moved on, and they'll effectively have to start over from the beginning. Ultimately, a contract-to-hire strategy overwhelmingly favors the employer's interest.

Interesting, I hadn't thought about it that way. I always prefer a month or two trial run before I commit to joining a company for the long term. I just can't tell enough in a few hours of interviewing whether I'll enjoy working with the people or not. And also you rarely get to meet all your future co-workers in the interview. But then, this has always worked out well for me. I could see it being worse and/or disruptive if it led to a situation of doing one or two month contracts for a really long time.

I think this is obvious, but look at the alternatives.

* Resumes can be exaggerated, falsified, or perhaps more often often leave out important details or relevant experience. Reviewers often also put unconscious emphasis on formatting or style (I had one colleague that I swear would accept anything in Computer Modern). * Take-home projects (if not carefully designed) can be a considerable time sink and reviewers may be biased toward particular coding styles or languages. * Prior employers are often not allowed to provide meaningful references, and even when able to, won't often have the time or motive to do so. * Referrals will be biased towards the friends of your current workforce, making it hard to improve the diversity of your workforce, and are a pool that will become exhausted as you scale or if your employees stick around. * Behavioral interview questions are biased towards candidates that know to prepare for an even more contrived corpus of questions with even less bearing on day-to-day job performance. * Candidates may not have prior code that their are able to share; certainly most employers make that difficult or impossible. Expecting side projects or open source contributions biases towards candidates that have means or desire to program outside of work.

I think it's ridiculous to think that a programmer should be hired without any part of the process evaluating their programming ability. The key is to do this while minimizing stress for the candidate. Explain the format of the interviews up front. Tell them about the people they are interviewing with. Let them have breaks. Help them through tricky parts if you have to. Avoid things that aren't realistic (whiteboard coding) but don't shy away from things that are (whiteboard a system design). If leetcode doesn't represent what you do at work, don't make them do it.

Given all the time I would need to pour into interview prep, take home projects are far cheaper. Leetcode isn't a time sink?

I don't see a bias towards language or code style as a problem as the work will be done in a particular language. Style can be solved with a style guide. Obviously they cannot be judged as harshly as in in company code review, but because language is a job requirement anyway and style can be a set standard, it seems to be better than most choices.

So the study seeks to investigate the role of an in person judge on the whiteboard interview. The study found that a private whiteboard interview, where the candidate gets the same problem but solves it without a judge in the room, was better at evaluating software engineering skill than the same question with a judge in the room. To your point, we can keep all the benefits of algorithm questions with less of the disadvantages by modifying the typical whiteboard interview to just remove the judge in the room.

>”What’s more, the specific nature of the technical interview process means that many job candidates try to spend weeks or months training specifically for the technical interview, rather than for the actual job they’d be doing.”

Had to go through half the article before this showed up. You could be learning something new and useful rather than spend so many cycles trying to master trick problems.

To me, the bigger question is, why do these interviews require every candidate to prove their programming/design skills from scratch? Say I have worked at a respected tech company for N years, my record there should provide a lot more signal than a day’s worth of interviewing would. I think we need some sort of standardized metrics for different dimensions of tech skills, and those metrics update over years of experience. Then companies don’t need to assess tech skills - they just need to look at your record, knowing that it is objective. Certifications, open-source contributions are some examples. The problem is the work you do closed-door in a company cannot be objectified or standardized easily. If we can solve that, that would go a long way here. Interviews can then focus on soft and behavioral skills solely. Not to say that those interviews are any more useful or signal-deducing as they happen today.

Will anything change in the near term (~2 years), I would bet not. I sure hope the way interviews are conducted changes. After all, most of us aren't trying to spend many hours on sites like leetcode.com. I find those sort of problems to be fun and enjoyable on my free time, but they can be VERY challenging under pressure.

Have you ever read Coders at Work?

One of the things that terrified me when I read it was Ken Thompson's description of his interviewing process.

Paraphrasing: He would ask about a project the interviewee was proud of, then drill them with critical questions about it.

I actually like the general idea. But the social pressure of this situation seems to be quite intense.

I have basically been doing a second job sending out applications and interviewing hard core for about a year now. There is so much time I have accumulated in take home projects. Anxiety is a big issue for me interviews(Yes, I am a woman) and even in ones that go well I am still clearly minorly affected by it. There was one back in November that I got to the stage of being flown out to their office. At the end of the interview during small chit chat I was nervously talking about the stickers on my personal laptop.(My plane had been delayed to weather and I was basically interviewing with them a couple hours after everyone would normally stop working.)

My count on applications is well past fifty and I have done so many phone screens(HR and technical) that all of them seem like a blur to me. At this point I wish I could throw a portfolio of my work at a company and get a job. From a Reddit thread about "unrealistic car mods" shows my overall skill set in my ability to get the job done. https://www.reddit.com/r/cars/comments/hdrgv1/car_people_of_...

My current job I got because I found a major vulnerability in the company's authentication system and recovered the private key for the client to server encryption.(I didn't make much money at the time and didn't want to pay the subscription cost for the premium service.)

I have interviewed/hired nearly every person on my current team. One of them was almost denied by a senior manager because the candidate wore a backwards baseball hat during the interview.(Lack of professional attire, etcetera.) The rest of the interviewers over ruled the manager saying that was a dumb reason. Turns out this candidate was super conscious of a balding spot on his head and he wore hats to hide it. He turned out to be an amazing coder and great at solving problems that came up.

For me, it's almost impossible to think in any kind of group situation. I do ok sometimes on interviews. It really depends. If given time alone, even during an interview, with my own tools and without distractions or having to tell the idiot interviewer what I'm thinking (as if anyone can think and talk about two different things, the problem and what the interviewer wants to hear at the same time), I do amazingly well. With another person watching, in-person or remote, I can't even think of how to start. I get caught in loops of anxiety and I can't relax enough to think things through. I also am afraid of trying wrong things so sometimes get paralyzed a bit. Mainly, I try to avoid such interviews and go for ones that will let me solve the problems on my own, with my own tools, without some idiot watching over me.

In one of my recent interviews at a mid size silicon valley company, I was given 4 hours to make a tetris like game.

One of the most enjoyable interviews ever with no whiteboard and no one around me to check on my progress. I was blank for first half an hour or so but made great progress by the end. Eventually got the offer.

20 years startup/internet engineering here. I have "failed" roughly 50% of my interviews usually as the result of failing the coding rounds. Almost all of fails were conducted by someone with nearly half my years of experience who were interviewing for something from the CS academic curriculum. When I am asked to conduct an interview, I don't do anything much more complicated than a glorified fizzbuzz. I prefer doing architecture interviews.

My biggest problem with algo interviews is most of them are trying to determine if you happen to be familiar with one particular algorithmic solution to something - if you don't happen to know that one algorithm, you fail. In the real world, you have to learn as you go. You never know the solution to every problem in advance.

The problem is we don't like the process, but not many people propose reasonable solutions. One extreme is just hard LeetCode questions, and the other is, "they should just talk to me about my experience and projects." We need something in the middle. Some ideas I've been thinking of:

- Pair programming with a problem that neither the interviewer nor the candidate has seen. Doesn't matter if they can't find the optimal solution. It would show how good of a communicator and how proactive the candidate is.

- Make the candidate implement a solution to a problem that's already solved. The interviewer shows the problem, explains the solution, and the candidate needs to code it. It would reveal the coding fluidity, understanding of specs, how they ask questions, etc.

Thoughts?

This is why I really go out of my way to try and dissipate the anxiety as much possible. And because I feel anxiety myself when I interview people, so I remember that we may share that perspective and we'd both have a better conversation if we could relax a bit.

My interviews therefore end up being conversational, and definitely non-confrontational. I want to hire someone who the team will get along with, who knows what they are doing, but in the end I have to remember that if we hire them I'd prefer we didn't start off our relationship with an over-the-top technical interview.

So far I've been successful at finding good candidates this way. Maybe it's just luck. But I don't think leetcode interviewing is the only way to get well-qualified employees.

As the interview process seems so crap and few people have any suggestions, here is mine.

I got asked to bring in my laptop with some code of my own we could discuss. I had a side project at the time so wasn't a problem. I got asked some questions, asked to add a simple feature. It was pleasant. I knew the code, so I was relaxed and didn't need to look up stuff. I think the interviewer learned some stuff as well.

The only problem people might say is that you used someone else's code. Even if that was the case, if someone else can talk confidently about code that they didn't write and jump in and add a feature easily, you have probably found someone good.

Can we try and push this as the new way of doing interviews. Low stress and respects peoples own time.

Now you want someone else to go through your code ? Without any idea about the mess you have written

There's one genetic factor known for the performance variance against the stress, which is also called "warrior vs worrier": https://www.huffpost.com/entry/stress-management_b_2671591

IMO, there's Moneyball-like opportunity for companies who can utilize (or, in another word, exploit) this aspect. A tech company may conduct tech interviews with a private environment & a traditional environment, and if one's performance is much better in the private setting, they might be able to offer a job with a reduce salary than the candidate's capacity shown in the private setting.

I get massive anxiety interviewing for tech companies, and I've been doing it for many years. The worst interviews are the "trivia quiz" questions that are either correct (according to them) or incorrect. There's no discussion of how things work and no frame of reference or anything. For example, when I interviewed at Amazon coming from Microsoft, they asked me lots of Javascript 'trick' questions even though I told them I was using C# day-to-day. I knew most of the questions, but got caught up on a couple of JS 'tricks' that I just couldn't recall at the time.

Sadly, it's all a game, but the good thing is that interviewing is a skill and can be improved with continued practice.

First thing I'll say is that I'm not an advocate for how industry often interviews today, whiteboard and puzzle problems and all, and their efficacy is worthy of discussion. It's a hard, hard problem and there are major deficiencies. But this academic "study" seems highly suspect.

48 subjects and we can make a generalization? People's stress reactions seem intimately tied to enormous number of contextual factors. Looking at the abstract it seems the "goal" of this study is "to make problem-solving assessment more equitable and inclusive". A noble and worthy goal but this does not sound like academics or science. It sounds like its conclusion was sought for.

The current "tech" job interview process is engineered to select for people that are both competent and secretly insecure about their skills, and to do this with little or no false positives. Competent employees are, well, competent and secretly insecure employees tend to have a need to impress. Both of these traits works to employers benefit.

A large amount of false negatives, like candidates that would otherwise be great employees but maybe just suck at whiteboard coding, is considered acceptable by most employers because research shows the cost of bad hires is much much higher than missing out on many good hires. Employers know the current process is miserable and leaves a lot of good people behind.

I can attest to this. A number of years ago, after fifteen years as an engineer getting jobs through my reputation, I went into my first white board coding interview, with one of the FAANGs. It was a Boggle like problem to find possible words. While at the board my back was killing me, and I had a horrible time thinking due to anxiety. I knew that I wanted a trie, but I blanked on how to implement it properly. A couple hours later at home I remembered all of it.

That said, it did filter out people who had trouble handling the new stress of a whiteboarding interview and people who did not practice whiteboarding. If I ever interview for a FAANG again I'll make sure to do a lot more preparation.

I think for senior level positions especially it is more important to see how a candidate approaches a problem beyond just memorizing algorithms or LC solutions. I wish there was way more emphasis on "case study" style approach where you could discuss at high level how to build X. Even pulling the candidate into a real technical discussion where you're trying to optimize something and see if the candidate has any input. Building up on experience is what we should be after.

To me, asking LC or the same algo and data structures questions (implement DFS .. wow genius) is just gatekeeping via a measure that most people can pass by just memorizing stuff for a month.

I think there are a lot of hidden biases in hiring that no one will overtly acknowledge e.g. personality, gender, attribute of race (yeah I know, but still there are people unaware of their racial biases), ability to take orders, if they are too good and might threaten their job or make them work harder because they will output more completed work, etc.

usually this line of thinking is summed up as the follow question: Can I see myself working with this person for 40+ hours a week?

then they bring down the axe with the following: not a good 'culture' fit and move on to the next candidate and the broken cycle continues.

This is all in addition to the broken whiteboarding challenges.

I'm susceptible to stuttering and/or rambling when I'm extremely anxious, sometimes without even being fully aware of it. I made it to final interviews for tech jobs twice, and I got the offer both times.

In retrospect, I was extremely anxious the first time, but the company was about to enter a high-growth phase and didn't seem to have fully-developed hiring practices (I got a tentative, very informal verbal offer on-the-spot, and a phone call to confirm I had the job about 24 hours later).

The second time, I was interviewing with a big US bank so I assume they had a better eye for details. Maybe I'm getting better at not showing my anxiety.

The root problem isn't the interviews themselves, but the belief that--despite a stunning lack of evidence--interviews are somehow useful measures of future job performance.

And I believe this applies to most job interviews, not just in tech.

Hmm...but if everybody is stressed out, then it equalizes and you're testing for skills again, just skills under pressure.

And having skills retained under pressure is actually a valuable trait in this industry. And at least at desirable companies, the process is designed to minimize false positive, it doesn't care that much about false negatives. This is by design.

My guess would be that they filter out orders of magnitude more people with poor resume-writing skills or that didn't go to the right school.

Although I do agree that interviews that you can/must practice for are silly and there's a lot oof that around.

In my normal work I don’t have someone staring at me and probably thinking I’m an idiot for blanking so it’s not at all a simulation of work stress

At Google I'm convinced our interviewing system only works due to a combination of luck, large numbers, and generally good intentions. None of the individual steps is a particularly good way to tell if someone will succeed at Google, but anyone who passes all of these filters is unlikely to be a terrible engineer.

When you avoid terrible engineers and hire by the tens of thousands, you get plenty of good engineers and a few great ones. You also miss a ton of qualified candidates, but Google has more of those than it needs. It can afford to miss them in a way that small companies can't.

I have found that what best predicts interview success for me is the ability to confidently give answers, even if they are nonsense. One of the problems is that interviewers often aren't sure of the answer either.

I work in an interesting tech career that does need to assess this: Pre-sales Engineering.

At my current job, we have our candidates do a short take-home assignment to assess coding skill, but we also have them do a live presentation to assess their pre-sales skills and ability to present under pressure.

We try to separate the two tests to get a better read on the two skill sets we're looking for. So far we've hired strong candidates. We prioritize avoiding false positives over false negatives; in other words, we prioritize turning candidates away over hiring someone we're unsure about.

I leave tech interviews feeling like I had just been through a hazing ritual.

I don't think it helps that what works for some people doesn't work for others. I coded very very little before I graduated college but got my first tech-ish job because I knew a guy and I could explain my thought process well despite having no idea what grep was.

A friend of mine would have fallen on their face during that sort of interview, but they're more book smart than me and write quite well.

If we were the two candidates there, I would have got the job, but they would have done it better. I don't know what the solution is, but it isn't more of the same

Engineers tend to over-focus on correctness. I’ve run hiring process in the past where over a year we interviewed around 50 engineers. Feedback from our own engineers involved in the process would often be like “Seems like a nice guy. But his solution to the challenge - one he’d never seen before - had a bug in it! And the solution was just O(2^n) and failed to notice the O(n^2) solution. Even though I glared at him a lot and kept distracting him with hints” ... I’m exaggerating for effect of course but some form of that would happen often

The best interviewers can feel you from certain job experience questions that you’ve done in the past. It’s looking for a series of related answers that a candidate just cannot make up on the spot.

So what are we going to do about it? Nothing. Absolutely nothing, that's what.

This happens all the time. People complain and complain, something comes out proving them right (or at least confirming their opinion) and all that happens is people relate. Politics, jobs, healthcare, interviewing practices, work conditions...

If programmers actually joined a union and organized themselves to make a db of companies with shitty interviewing techniques, bad business practices, etc. maybe something would change, but hey, complaining is easier, right?

At one company where I was the interviewer the company used a few simple java questions for each screening (not great I know). It was basically just a way to make sure they did have experience with Java at at least a junior level. This became an issue when we started getting candidates from recruiters. One candidate bombed on the first question. Then when we asked the second question they answered the 3rd. Clearly the recruiter was trying to coach the candidates based on past questions.

The strangest thing about jumping through big tech corp take home projects and multiple interviews is they then place you in their internal training boot camp and you realize why didn't you just stick me in here in the first place then rank my results when we finished training instead of wasting money for a week shopping me around departments so highly paid senior employees could all stop what they were doing just to have a go at me with the whiteboard and multiple on site lunches.

This makes sense if you’re at a FAANG organization you shouldn’t need to make breakneck code changes with people breathing down your neck.

At a lean startup or smaller team though this isn’t unheard of. I’ve had to debug and code up multiple things while everything was on fire around me.

Mistakes are guaranteed to happen no matter how many precautions are in place. You can reduce the probability of a mistake happening but when it does whoever is fixing it needs to understand how to stay calm and work well under pressure.

Most interviews I've done in my software career were done on more than 24 hours without sleep.

Not by choice, but because I cannot sleep the night before an interview. I have difficulty sleeping in general, and the anxiety of an interview drives it through the roof.

Whiteboarding is also terrible, pretty sure it is used intentionally to skew hiring results toward certain types.

If someone is looking over my shoulder while coding I can't think, I think best when there are no external inputs(no noise/eyes closed).

Idea:

Assign a short coding challenge and rank candidates on their solutions.

Then, one at a time, hire each candidate as a contractor for 20 hours of work. Let them fix bugs and do code reviews. See how effectively they communicate through code and in code reviews. Invite them participate in the weekly standup meeting.

Finally, have team members who worked with the person write confidential feedback with a hire/no-hire rating and justification. Hire the first candidate who gets all 'hire' ratings.

When you're under time-pressure, you actually spend more time thinking about how much time you have left, rather than thinking about how to solve the problem.

It's like that parable.

2 students approached the master, wanting to become swordsmen.

'I want to reach your level in 10 days', said a student. Master replied 'It will take you 10 months'. The other student said 'I don't care if it takes 10 years, Master'. The Master said 'and you will learn in 10 days'.

I find it interesting that academia often points out that skills interviews have a soft-skills bias, but rarely do I see them consider that this may be a feature and not a bug.

> “Our study suggests that a lot of well-qualified job candidates are being eliminated because they’re not used to working on a whiteboard in front of an audience.”

This is a common daily task at any company I've ever written software at. People who can communicate their ideas effectively are better at their job.

Right? At least pre-covid, I would say I spent at least an hour (sometimes many hours) every working day "explaining things on a whiteboard in front of an audience". It's a huge part of my job.

(Post Covid, I do the same thing, but have to do it over chat)

IMO: "Do you have a clear understanding of your area of expertise, and can you communicate that understanding on to others?" is a far more useful predictor of success than "Can you find the right answer to the kind of problem that can be solved on a whiteboard in 45 minutes, or in 5 minutes by searching for an answer on stackoverflow?"

That said, I agree that generally interviews are terrible. But the (implicit) screening for soft skills isn't the terrible part.

I've had the opposite experience. In >10 years of software, I've never written code on a whiteboard outside of an interview. Systems layout, designs, code pathing, sure; but never real, compilable code.

I agree with the last point, but I don't think either of us can tie our anecdotes to it.

There is something to be said for the soft-skills evaluation being relative to the expectations of the job position, sure.

I'm saying: I find it odd that I rarely see these types of studies discussing the merits of that nuance. Especially when the idea of differentiating between soft and hard skills is not something that would be foreign to the average interviewer.

"Interviews are 100% for soft skills and technical evaluations are 100% for hard skills" seems to be a common assumption which I'd argue is false. Few tasks a developer does is purely a hard or a soft skill, there's always some overlap.

This problem is perfectly clear to people who have had senior level careers in any other professional industry. Software does not know how to assess people because it lacks ethics and standards.

It’s a problem that most people seem to not want fixed until they are themselves negatively impacted. Since the very concept of ethics is so utterly foreign its need and application cannot be effectively communicated without resulting in some manner of embarrassment or hostility.

As suffering from anxiety disorder, It totally hits home. I have been a top notch employee at every place I worked, but I totally bombed most of my interviews.

> In short, the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding.

In most of my programming jobs, explaining your work was most of the job and writing code was less important.

The whiteboard environment is a bit unnatural but I don’t want to hire someone who gets the right answer if they can’t explain it to me.

Most of the job: was that explaining while you write the code, under time pressure, or explaining your debugged code after you've tested it a bit and have confidence in it?

> The whiteboard environment is a bit unnatural…

So have them explain it to you after they complete the problem with you out of the room.

If I’m in the room, I will steer them away from dead ends and ask leading questions when they get stuck. The point of the interview is to give the interviewee as much of a chance as possible to demonstrate their skills.

I’ll say for sure, not everyone interviews this way. A lot of devs running interviews think of it as a “test” with pass/fail. I’d say the better interviewers see it as an opportunity to dig around and figure out how to get evidence of the candidate’s skills. That means being creative, that means responding to what the candidate says and finding opportunities to focus on their strengths.

The problem is that there's lack of transparency in the process. Candidates don't know the "real" rubric that interviewers use, so anxiety could make the situation far more antagonistic than it is. And to some point, it is inherently antagonistic; the interviewee is being judged, vulnerable to being rejected, by someone who may or may not be unfair. So the anxiety ratchets up. This can lead to all sorts of bad behaviors like candidates not asking questions for fear of revealing ignorance or looking stupid. And a lot of interviewers as you say, especially at the top firms, don't really care about "how a candidate thinks" at all; they want to see correctness, they want to see efficiency, they want to see whiteboard code that can compile.

I don't think this necessarily means the "answer question alone and present answer after" approach is the right solution to interviewing, but it does show that interviewers might need to take some steps to make the process less opaque and daunting to lower candidates' potential anxiety.

> And a lot of interviewers as you say, especially at the top firms, don't really care about "how a candidate thinks" at all;

From my direct knowledge of some “top firms”, if you only cared about correctness then you wouldn’t be able to fill out the interview feedback form.

Transparency may help, but in my experience, that’s what recruiters actually do and some of them are fairly good at it. The recruiter has a strong incentive to get you hired (they get bonuses for hiring people!) and will explain as much of the process as they can, if you are willing to listen.

That said, there seems to be a fair bit of turnover in tech recruiting.

And, as you said, every interviewer will be different. But I don’t see a way to fix that without eliminating technical interviews altogether. You are being judged, the interviewers are subjective, they do disagree about the rubric, and a bunch of money is on the line. It’s completely reasonable to be anxious. I don’t really see a way out.

> And, as you said, every interviewer will be different. But I don’t see a way to fix that without eliminating technical interviews altogether. You are being judged, the interviewers are subjective, they do disagree about the rubric, and a bunch of money is on the line. It’s completely reasonable to be anxious. I don’t really see a way out.

Perhaps what we should be working towards is making it more transparent- it's still ubiquitous for candidates to be rejected without feedback- and more objective, and improving the technical interview process further. I'm not arguing for throwing out the baby with the bathwater.

There is a world of difference in explaining your work to someone you've never met before on a problem you've never seen in an environment you've never worked in.

Versus explaining your code to your coworker whom you've worked with for the past X time who understands some of your eccentricities or idiosyncrasies and without a pass/fail judgment lingering over your head like a sword of Damocles.

I would definitely not want to work for you, I think there's something deeply ineffective when most of the job is explaining your code.

I say that as someone who cares a lot about writing easy to read code and has no problem explaining it. Also, I'm very bad at solving a problem while being required to think out loud.

The skill of explaining your work is a very important one in practically any creative problem solving field. This only increases as the technical complexity of the problems faced increases.

Sometimes this explaining happens in textual format, like code reviews and design specifications, but often it happens verbally. Absent an explanation, it's hard to build consensus among the parties involved (co-workers, users, management) that your solution (as manifested by the resulting code) is the right one, and for anything non-trivial, you surely can't expect others to just assume that it is the right one.

explanation is important but its unreasonable to expect someone to do it on the spot in a time-pressured environment. Often you have to go away and think about it alone in order to come up with a good way of explaining things to others.

> Also, I'm very bad at solving a problem while being required to think out loud.

There’s a simple strategy for this, which works fairly well in both interviews and meetings. You give people time to focus and think about the problem and solving it, and don’t force people into discussions right away. It’s not a panacea but in practice it is very effective.

The reason why this is important in meetings, as someone running a meeting, if you don’t give people time to think, it will always be the same one or two voices in the room. This is why brainstorming sessions suck. The same thing applies when you are choosing who will take responsibilities—the same few people always speak up first.

> explaining your work was most of the job and writing code was less important.

That's not what is being asked. It's explaining your thought process while actively doing the work in an "unnatural" environment.

> The whiteboard environment is a bit unnatural

I don't know anyone that codes on a whiteboard.

> but I don’t want to hire someone who gets the right answer if they can’t explain it to me.

Again, that's not what is being asked. They are being asked to explain their work before they get an answer.

> That's not what is being asked. It's explaining your thought process while actively doing the work in an "unnatural" environment.

In short, that’s a limitation of the study.

The study is comparing “explain your thought process aloud as you solve the problem” against “solve the problem by yourself and don’t explain anything”. There’s a whole range of sensible middle options, like “solve the problem by yourself and explain it afterwards” or “solve the problem with the interviewer and then explain it”.

I sincerely appreciated that the technical portion of the interview for my current employer consisted of pair programming with their developers to improve on the pre-screening "take home" assignment I completed. It actually gave me a chance to explain my thought process and work through problems in a similar way to how it happens on the job. It was infinite preferable to white board algorithm questions.

Nothing will change.

I think the problem often lies with the interviewers’s ego that results in this strange paradox where they’re trying to demonstrate their competence to the interviewee by way of technically “outwitting” them.

That way, in the event the interviewee is personable enough to get hired, the working dynamic is that much closer to being formed between the two, and chances are reduced that the interviewer is hiring a potential rival.

I shared this article with my company. It was amazing how fast our leadership (particularly those involved in interviewing) came back with critiques about sample size or if they properly accounted for student performance in the study. Perhaps fair critiques, but it's not like this effect is unknown by software engineers (unless your head is really stuck in the head). Ego defense at its finest.

A lot of it relates to how much the interviewer can create rapport and comfort. The interviewer having the upper hand in the power balance, they should go the extra mile to set the stage correctly. Many good engineers are terrible at interviewing. In the wake of the interview they boldly set forth their assessment while being largely ignorant of their own negative impact on what just happened.

How many years took them to realize this?... As someone who spend like 6 months job hunting and doing more than 20 interviews(including Google, Microsoft, etc), this was pretty evident. Yeah, I forgot to code that weird problem which implies the use of dynamic programming in a white board, but hey... what about all those years of experience and evident code skills. It is ridiculous.

Occasionally I work as a speaking examiner, assessing people's foreign language level.

The test is rigorous and well designed. Few minutes of questions are designed to be so easy any can answer them. "How do you spell your name?" etc. This is by design as people can be extremely nervous when taking a foreign language exam.

If you are interviewing people, do what you can to put them at ease at first.

Totally true that people with no self-doubt/impostor syndrome overperform their ability level in whiteboard interviews compared to people with more self-doubt, who are often the better engineers. I am personally a beneficiary of this. It also contributes to all of the terrible dick-swinging culture in a lot of tech companies since these are the people who pass the interview.

Its really funny. I was interviewing at a large company recently and the way they do interviews is weird. Each team that is interested in you does their own on-site. The first one I bombed b/c I was so anxious but after I experienced it once, I aced all the other teams I interviewed with. Nerves definitely play a role, its cool to see that it is being quantified.

Sounds like Apple. Same thing happened to me :)

I always care about explaining to the candidate how the interview will look like, and what are the most important elements I care about. When you skip this part, the candidate has no idea if for example during live coding, you care about the results, the code quality, IDE shortcuts fluency, etc. The feedback after the interviews proves it is a game-changer.

I used to be super anxious and get brain freeze.

This doesnt' happen to me after I grinded leetcode for a couple of months.

you are anxious if you feel unprepared.

I do think this is true - but I also think that the ability to control anxiety (an increased tolerance for uncertainty) is a characteristic that is learned with experience and can be a marker of an experienced engineer. Someone who panics in an interview might also be inclined to panic when a manager frantically reports a bug in master, for example.

I've posted this before on HN: Tech interviews are designed to find people who are happy in a role where their level of responsibility begins and ends at: "Here's your coding assignment. Go do it". The interviews are designed to eliminate people looking to get in the door, then quickly move into or up to a non-coding position.

This should hit home with everyone. What I would add is one thing that adds to the anxiety is whether the interviewer is looking for reasons to _reject_ a candidate (i.e. don't let the bad ones through) vs reasons to _hire_ a candidate (i.e. let them shine on aspects that are hard to communicate on a resume or when checking references.

Anxiety management is absolutely foremost in these interviews. I did not successfully get through one of these interviews until I started experimenting with taking anxiolytics beforehand.

I eventually found that a moderate does of phenibut 5 hours prior was an ideal balance of social inhibition vs the types of impairments that come with these drugs.

As someone with bad anxiety and who gets panic attacks in roughly one third of panels, it does feel fairly discriminatory.

The best interview I had was when they asked me a whiteboarding question, gave me a pencil and paper, and left the room. I solved it, they came back, and we talked about my solution. They still got to hear my thought process.

I remember interviewing at Google and being completely unable to think or relax. Instead of making any effort to learn about me and my skills, the interviewers became more aggressive and nitpicky.

Looking back, my life is dramatically better working remotely for the same level of income with friendly helpful coworkers in a non toxic environment.

Last year, I went to both Amazon, FB on-site interview at Seattle and Vancouver office, and completely bombed it (not quite, I solved 50% of the programming questions). I felt like a fraud when try to answer the LP questions. Anyway, is there any companies other than FAANG that can support relocation (to US or Canada)

But did they see if being neurotic is a net positive or negative for doing the job. Skill isn't the only variable in being good at a job.

If you skilled but you neuroticism makes you less productive even compared to someone less skilled who cares about skill level.

You could end up with a skilled team that folder under low amounts of stress.

I have totally blown interviews and also done very well. To me, the interview is a distinct skill set. You have to treat it as it's own thing. What matters is practice, practice. Not a little bit. Say 6 weeks. Then you are ready to slay. When I haven't done that I've been rattled and done poorly.

Suppose you're BigTechCo, and your interview process passes up this huge pool of talented developers who don't interview well. Then your competitor ScrappyStartup, who gives some of these folks a chance, ought to be eating your lunch, right?

Something to think about, especially if you're involved in a startup...

To a first approximation, that conflation is fine; software engineering is stressful work where the tasks to be solved often involve known and unknown unknowns, and it's important to observe how a candidate performs under pressure because the job will include pressure.

Looking at the article, it seems all they looked at was whether the code worked or not.

interviews also assess communication and thought process. and working code isn’t necessarily good code. also wonder if the people in the private room used their phones to research answers

To be fair, even these interviews are mostly garbage, they don’t have a better way to gauge you with just 2-3 meetings. Which really sucks. What they should do is to check your pass job experience, verify from the people you work with. That’s not easy.

I feel like sometimes big tech companies give technical interviews when they're not actually hiring, that it's a sort of formality. Can't really confirm it, but thinking this way makes me feel better about not passing to the next stage.

Correct. In many of these companies, part of your evaluation is how many interviews did you give. This means: the game is to do as many as these interviews, typically once a week, such that you are not penalized for doing too few interviews.

I was once penalized for giving too few interviews but there weren't enough candidates at that time! While I cared about giving every candidate a good chance for success, I can say confidently that not many did care. They only cared about getting credit for doing the interview.

I didn't really want to speak up on this but yeah this has been a major problem for me. I have a disability and diagnoses with anxiety disorder. These intensify the symptoms even more and sometimes I blank out.

I failed every single interview when I had to do programming task on the spot, however people who hired me based on my resume and how I talk about solving problems, had great business with me for many years.

This was interesting:

For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed.

That is an amazing statistic! From 100% failure to 100% success!

It's important to bear in mind the small sample size: 4 women took the private interview, and 6 took the public.

Still a compelling result, but not quite as incredible as it sounds in the article. I'm not sure what the p-value would be.

Yes, I don't think a company can continue with whiteboard interviews anymore and consider themselves to be treating women fairly in the hiring process.

I haven’t interviewed for a job in nearly 19 years. I think job interviewing has changed a lot in the intervening decades and I’m a bit worried about how I’d do if I ever need to interview again.

Technical interviews do not resemble any kind of professional software engineering environment that exists in the real world. That is their biggest problem. Everything else is secondary.

Get a whiteboard at home - even a small one. Try to write an algo on it. You will be amazed at how stressful it is, WITHOUT anyone looking over your shoulder.

But with practice, that stress goes away.

Direct link to the paper: http://chrisparnin.me/pdf/stress_FSE_20.pdf

Agreed! Cut throat interviews in which you are expected to perform under heavy scrutiny can heavily affect performance. I prefer the case study approach instead!

I fear there's a movement to get rid of leetcode interviews and I'm against it. What's the alternative? We go back to "you have to look the part", nepotism, credentialism, and other such BS in other industries. A data structures and algorithms test is fair and objective. You can objectively get all the challenges correct and then there's little reason to not hire you. Compare that to interviewing for, say, an investment banking position where you're judged on stuff like what frat you were a member of and how charismatic you are.

> A data structures and algorithms test is fair and objective

The article makes clear that the issue isn't the content of a technical interview, but rather how it is administered. Specifically, from Chris Parnin, a co-author in the study:

> But the format may also serve as a barrier to entire classes of candidates. For example, in our study, all of the women who took the public interview failed, while all of the women who took the private interview passed. Our study was limited, and a larger sample size would be needed to draw firm conclusions, but the idea that the very design of the interview process may effectively exclude an entire class of job candidates is troubling.

So the "objective" in-person tech interview excludes a huge bunch of talented applicants because there is nothing objective about it at all.

One alternative is providing candidates with a problem to work on for a few hours on their own (have them bring their own laptop, allow them access to Google and their own development environment), then have them walk you through their solution.

Do this as part of the interview and not something they have to do in their "spare time" at home. It's not uncommon for large companies to do all-day interviews anyway, so a way to structure this can be to have the interviewee come in in the morning, provide them the problem and allow them to ask any questions they might have (as if it was a real work problem), then allow them the morning to work on it.

Then in the afternoon you can review the solution and ask them to talk through how they arrived at it and what their thoughts were during the process.

The study in the article suggests that the same whiteboard question done in private rather than with a judge in the room has all of the benefits with none of the disadvantages in terms of the candidates software skill.

There are more alternatives than "look the part" and "leetcode". If I'm hiring a software engineer, I'm not hiring for how they look - the compiler doesn't care. And I'm (probably) not hiring for the ability to solve leetcode problems, either - of the work I would assign to someone I'm hiring, almost zero of it looks like leetcode problems.

What I need is a way that tells whether the person I'm interviewing can actually do the work that I'm hiring them to do. Maybe I should ask them about that. Maybe I should give them some problems similar to the actual work.

> What I need is a way that tells whether the person I'm interviewing can actually do the work that I'm hiring them to do. Maybe I should ask them about that. Maybe I should give them some problems similar to the actual work.

Would you mind sharing with us how to draw the rest of the owl?

Look, I don't even want to defend leetcode here, but it is a struggle to find questions that are:

1) Correlated with the actual work performance

2) Neat enough to be explained and solved in the course of an interview

3) Have objective enough answers so that it's possible to overcome likability bias in interviewers

4) Have objective enough answers to be able to meaningfully compare different candidates

5) Demonstrate that candidates can actually go from hearing a description of the problem to producing code that solves the problem

The best I've ever seen is the way my current boss does it. We do a two-hour interview (they've already passed a phone interview before we bring them in). We spend the first half hour just talking to them personally.

Then we give them a piece of code on paper. It's fairly simple - half a page. The function name is foo(). It's written in C#, but we don't expect them to actually know any C# details. The function takes an array of Strings as input, and returns an int. The questions are, what values does it expect for the Strings, what value does it return, what would they name the function, and what can go wrong (what exceptions can it throw - but we don't expect them to know the details of C# exceptions, so what might cause exception-like circumstances). This shouldn't take more than half an hour.

Also, it's the same code for every candidate, and we've seen a few dozen over the years, so we are fairly calibrated on how a quality candidate should do.

Then we ask them to write some code on the whiteboard. We give them a simple problem (again, same problem for every candidate). They can use any language they want. We don't expect them to be a compiler - we don't care if they get the parameters a bit wrong on a function call, as long as the code would be sound if the system accepted the syntax they expect. Again, this should take half an hour.

Then we give them a simple design problem. How would you design software to do this? It's a problem that is not rigidly specified, so they can have to flesh out the requirements (hey, welcome to the real world). It's got enough corner cases that we can ask them about one that they skipped, and watch them try to modify the design to handle it. We don't expect perfection - there is no perfect answer for this question. But we want to see them have some kind of reasonable taste or sanity in their design, and some ability to iterate the design as more requirements emerge. Once again, this takes half an hour. They're not done with the problem at the end of half an hour, but we've seen enough to know how they did.

Other than allowing them to ask any questions they want, that's it.

Thanks for the answer.

> Then we ask them to write some code on the whiteboard. We give them a simple problem

Can you share what kind of problem?

This actually sounds pretty close to what we did at my previous company, except that candidates were given a computer (nothing had to compile, it's just that we don't expect anyone to be able to write code on whiteboard) and two coding problems.

First one was pretty simple, such as printing a folder structure with nice indentation (given some predefined input data structure). The second one was harder, although we hired plenty who did not manage to completely solve that one, but if they did it was a plus.

Also had some design questions after that. Some people still complained that the interview was abstract algorithmic nonsense without relevance to real life.

It has some string splitting and manipulation, but nothing that couldn't be done in C. (It would be a lot more tedious, though...)

I don't want to give the exact problem, just in case we interview an HN reader.

> A data structures and algorithms test is fair and objective.

Anything but! For starters, in 2020 brute-force solutions are not acceptable anymore. Then, there are often several working approaches and substantial percentage of interviewers will fail you just because you use the approach they are unfamiliar with.

But even if you implemented the optimal approach, if the interviewer disliked you as a person then they find a way to fail you regardless. "Sloppy code", "bad communication" - no fairness and objectiveness in those assessments.

A juggling test would also be fair and objective. Just hit LeetJuggle for a few months before the interview and you’ll be golden

I think we need to come up with effective alternative to whiteboard interviews. Who’d be interested in forming a working group to come up with something?

Ok, so what can I as an interviewer do to reduce stress for the candidate?

Let's make good use of this discussion and compile a list of practical suggestions.

I'll start:

- Offering candidates their choice of whiteboard vs take-home. I've found that different people have different preferences, and they liven up when they are told they have a choice in the matter.

Can I pose a question to people here.

Would you have moved companies earlier or moved more often if a coding interview was not standard practice in hiring?

Managing your own emotions is a part of EQ. Why should we hire people with low EQ to a job as collaborative as software engineering?

Control is not a part of EQ. Awareness is. Being able to point to the source of your emotions is key in being able to identify the source of others' as well.

Fear responses are deeper than conscious thought. You can't expect a person to respond calmly to a life changing event like an interview anymore than you can expect them to stay completely calm in a locked coffin.

Software engineering jobs typically do not have situations that are anxiety-inducing like job interviews.

Unfortunately corps understand this and assessing anxiety is the purpose. Experience is more indicative of technical capability.

Just curious does anyone knows or has created a list of companies that gives you take home tests I would love to find out

Fizz buzz/leetcode != Real software engineering

I think we should come up with new software engineer interviews techniques

I always had the feeling this was the case, so my strategy has been to gaslight the person interviewing me.

I love how so many are passionate about this subject, always comments into the hundreds.

I've written about this before.

I've become pretty much convinced that many of the leetcode tests are "young-pass filters."

I recently saw an ad for a two-month, intensive course on how to pass interview questions.

The heck with that. I want to learn relevant stuff.

If people don't want me working for them, then I'm not particularly interested in working for them. Too bad. We could have made beautiful music together. I'm no slouch.

I usually just imagine visiting some friends that I haven’t seen in a while.

Well that sounds good to me. Sofware work can be pretty anxiety inducing.

The tech industry, if it can manage to do it, would benefit immensely from enacting interviewing methodologies that aren’t ultimately rooted in an underlying need to haze others. It’s as if the CS nerds forgot that when the frats hazed others we were disgusted by it.

What causes anxiety is wasting numerous weeks of your life that you could have used to pay your rent or spend with your friends, family or do something meaningful with your time. That's how much this people think you're worth!

Tinder generation, swipe next.

Social engineering like it's 1000 AD pre technology era.

No matter how difficult a coding interview was, the job afterwards was always harder and more challenging - taking that into account I would rather for even harder job interviews to reflect the actual difficulty of jobs.

I stopped applying to FAANG because of the exact reason.

The comments in these threads must give tptacek PTSD.

again, this demonstrates that interviews in tech sector are all wrong.

You don't ask plumbers to show how they do the job before hiring them.

I think this is known for years. Is't it?

It's been known, but there are a lot of naysayers as shown by people who still conduct this type of interview.

Science serves as definitive evidence which side is right or wrong.

Uh, maybe at times, but I assure you -- I've interviewed some VERY confident, VERY incompetent software dev candidates.

You're conflating the personality traits of the interviewee with criticism of the generally accepted 'best practice' of interviewers.

On my experience far too many companies are using processes that select for entirely the wrong criteria.

I think it’s pretty funny that HN constantly promotes articles about how shitty tech hiring is, and the threads are always a mix of workers agreeing and then managers often also agreeing but 100% confident that their hiring process works well and weeds out enormous numbers of faking bozos. It’s like, ok but almost none of you have useful actual data to back up your process and you can’t all be anywhere near as good as you think if hiring is, in fact, broken.

Same here. A guy was soaked in arrogance even though the test he submitted was average at best. "This looks good enough. when do I start?"

Even worse: I had an encounter with another guy who refused to make a test because his CV demonstrated everything we needed to know and asking him to solve a few problems was "incredibly disrespectful". He was also apparently going to start next Monday. Needless to say he didn't.

Tech interviews are an evergreen source of delight /s. I have been the interviewer and interviewee plenty. A few reflections.

- I try hard, and I believe with a certain success, to follow a sound approach to interviewing. I read the resume, I see where their contribution to the team I am hiring for can come from. Then, false positives and negatives never go to zero, and such is life (although some people are much better than others at observing and judging). I mean, people get married and divorced in 6 months, and it follows that there are inevitably bad assessments also in tech. But I try to ask questions that can allow me to gauge the skills of the person in front of me for the job that they are hired for. A conversation, more than a test. Things that can get googled are of no interest to me, for example. Or theoretical and never-to-be-used-IRL knowledge. Can you imagine do a try-out for a hoops team, you are a guard, and they try you out as a play? Or you do a try out for the Patriots as a quarterback and they test you on punting? Which is what happens in tech more times than expected by chance.

- Among the most nightmarish experiences I had as an interviewee, I remember: a hiring manager yawning for the whole interview; another time, I successfully (brilliantly?) completed a coding test in a few minutes and the interviewers said it was all great, I have no other questions. But then the recruiter told me that no, the interviewer felt that coding was so and so (what?). Another time I was dismissed after the first of five planned onsite interviews. No idea why.

- What about homework, you may ask? I took a few (2 or 3) homework for some of the most popular companies in SV. Well, one time I made this model and I don't know, maybe I will die an Achilles death for my hubris, but I believe I can fit a tree-based classification model after 20 years of work. The recruiter said that both graders like my model, but I did not do hyperparameter optimisation. I told them that the optimisation was both in the documentation of the homework and the associated Scala code. They said they were asking the graders again (it was just one, the other copied what the other wrote) and that they would have been back with an answer in a week. After a gentle nudge, they said that no, the graders did not like it anyway (they never took a second look and I wasted my time: for what, exactly?). I am not gonna take any other homework from any company, if not for reasons of desperation or destitution and I hope it won’t ever happen.

- Most companies rarely if ever do any business or technical analyses of their interviews. Google pollinated the tech world with their interview strategy and the other companies said: if Google interviews this way, why should we do any different? I work for a big company now, there is no post-mortem, and neither pre- nor middle-. We hired more than 100 people this way for a new team, it could have been better, worse, who cares? Nobody checks anyway. Because, and this is the main insight I’d like to present, especially for big companies (for start-ups it is different, an exceptional IC can “make” the company), it does not matter. There is a big area of 20+-D skill-space that is “flat” with respect to the employer’s contribution to the company. You don’t hire that uber-incompetent and you are doing fine because the market, overall economy, luck, people skill of the C-suite are much more important than ICs, managers, or directors’ skills (beyond a certain threshold, of course). Google prefers to hire talented people and then it does not allow them to work on anything crucial because the most important thing is not letting them work for other companies? It does not matter. And if the question is: but if the interview process is inefficient, are we not wasting thousands and thousands of employee-hours? I have got the same answer for you: it does not matter, because there is often no need (in big companies) for more than 20% of the time of the employee, people get hired for all sort of reasons, but rarely for (true) technical or business reasons.

does it mean that take home interviews as the best form of interviews ?

One observation of mine is that interviewing is often more about finding the most feasible and consistent process rather than the most effective one. Whiteboard interviews make for consistent evaluation and it's easy to train up new people on a set of questions. Scoring based on how far the interview got, and the level of optimization of the approach used is relatively unambiguous. So it's logistically easy and less subject to bias - that's why it's popular even though it has little resemblance to day to day work.

A more effective interview would involve tasks that more closely resemble day to day work. However, the examples of this that I've seen so far make for much more ambiguous evaluation or are much more difficult to schedule:

One example interview process I've seen is having people submit PRs to a codebase to fix bugs or implement features, and then review a PR submitted by the interviewer. This more closely resembles day to day work, but has the disadvantage of spreading the interview process across multiple days. Additionally, you might tell the candidate that they shouldn't take more than 2 hours to implement the PR but who knows if they spend 10+ hours on it.

Another alternative interview format is a 2-hour long one that starts with the interviewer asking "how would you build a text editor?" There's no right response. If the candidate gives responses like ropes and gap buffers then the interview might go in a direction focused on data structures and systems. Some candidates ask if it's a WYSIWYG editor like word, or an ASCII/unicode editor like Vim. In that case the interview might test the candidate's abilities to think through how to build an interface to decouple the UI and underlying data structures. The candidate might start off with a simple array, and work through why it becomes infeasible at larger lengths and think through ways to mitigate that. This interview was flexible to test technical knowledge and reasoning for candidates at any level, and could go in a variety of directions. But on the other hand that makes consistent evaluation difficult and training people up on this interview similarly difficult. This was at a small company with maybe 30-40 engineers, where each team basically had total latitude on how it carried out its own hiring. That's not how a lot of larger companies' interviews work, which often emphasize consistent evaluation and multiple interviewers.

The only interview which does more closely resemble real world tasks, and I suggest that more companies employ is debugging interviews. It requires the candidate bring a laptop and that the company build a couple repository templates, but once that's done it's a very easy interview to conduct. Just observe the candidate debug and make note of how many bugs are fixed and whether the fixes do bad things like breach layers of abstraction.

I wish I had an agreeable way out of this field. I'm not going to critize the current interview process, it has its advantages and disadvantages and theres a reason it's used, but if its shown me anything it's that I chose the wrong career.

It's interesting to me that the interview process of our field makes you question your choice of career. It seems like such a small thing compared to everything else.

I absolutely despise the interview process, but there's no question to me that I choose the right career.

It's just part of it I suppose. My particular strengths and abilities aren't what companies are really selecting for. This type of interview process just kinda supports that. Of course things are more complicated than that, like I said though I just mean it's a component.

> the findings suggest that companies are missing out on really good programmers because those programmers aren’t good at writing on a whiteboard and explaining their work out loud while coding

Sure, no love for whiteboards from me. However, this does suggest an arbitrage opportunity for an employer who is willing to look differently at these candidates, and then profit handsomely, Moneyball-style.

There's a bunch of money just being left on the table and nobody is picking it up, really?

My two cents is that interviewing candidates can give a whole lot more insight into how to perform in an interview. Interview performance is a skill that is separate from the actual job, and gauging that performance is sub-optimal. But everything in this space is sub-optimal, it's all pick your poison.

Having capital or hiring authority doesn't correlate with willingness to try to actually hire value rather than assemble something that they think resembles a team.

What I don't get is why no one is attracted by the social brownie points they'd end up with when their moneyballed team inevitably turns out to be 70% female.

Many smaller companies do pick up the money, which is one of the major factors allowing them to succeed in a world where the big software companies toss around ridiculous amounts of compensation. The Googles and Facebooks of the world are structurally unable to do the arbitrage because many (probably most) of their interviewers don't particularly care about interviewing.

Maybe some people (how many?) fail because of anxiety, but on the other hand, anybody who doesn't fail certainly has proven some minimal coding skills?

I usually took it as a positive if companies did some basic coding skill test, showing they at least cared that people can code.

Also, the people who fail because of anxiety, how will they fare in the normal job situation when they are supposed to discuss stuff on the whiteboard with colleagues?

Maybe the results could be taken as "give people who fail the coding interview another chance", but I wouldn't take it as "don't do coding interviews".

"Technical interviews are feared and hated in the industry, and it turns out that these interview techniques may also be hurting the industry’s ability to find and hire skilled software engineers."

It doesn't hurt "the industry", there's a competitive edge to hiring otherwise superior engineers that are passed up by poor hiring practices at certain companies.

Not everybody needs to work at FAANG, or worse, companies that act like they're FAANG. Less money, less bullshit.

I just went through a job search and this is what I landed on.

Every CRUD webapp startup thinks they need FAANG-like hiring processes and fucking suck at it.

I ended up taking a job that was mostly a no-interview process. Two calls to talk about the fit.

They looked at my job history (gasp!) talked to some references (double gasp!) verified my ability to code with my github contributes (omfg I can't breathe!) and offered me a position.

Less money, but definitely less bullshit. It was a good choice for me.

Will love to know the company name !

In tech cities almost every non-tiny company has the FAANG approach

If the results of this experiment are true part of winning the game may be taking some anti-anxiety drugs before the interview... I heard high blood pressure medicine works.

Anybody have any first hand accounts on using drugs to calm anxiety before the interview?

Very true. I suffer from this myself, and I have interviewed for Google twice. When I interviewed the second time I thought I bombed, but I was hired. With the benefit of hindsight, I can rationally tell myself that me getting into some pretty high flying places is not a coincidence, but I'm 1/3rd of a programmer in an interview compared to when I'm not under stress.

Well they are auditions rather than interviews. So it seem to be accurate.

Current SWE interviews seem extraordinarily effective. They've produced corporations that dominate our lives, with systems that have unimaginable capabilities. Their profits currently dwarf other companies such that the S&P500 is basically a tech company index. More companies will adopt hiring practices like these as the applicants become more competitive. SpaceX has grueling all-day multiround tech-like interviews like this, whereas Boeing just does 1 hour with 2 managers. The effect on employee quality is clear.

There's a phenomenon where an entity which is successful for reason X is unaware that it's hurting itself for reason Y, and may even attribute its success to reason Y. Which is all very well until X shrinks or Y grows.

The point is that you can't assume that just because something is doing well, everything it's doing is perfect. In fact, success is likely to hide all sorts of bad practices, because the bad practices just don't hurt enough to offset the success coming from another source.

I kind of suspect that this is why declining tech companies often crash and burn so dramatically; they tend to have a lot of myths that they believe about themselves, and when the one thing keeping them going declines, they can't address all the things that are pulling them down, because they don't really believe in them in the first place.

I'm about 99% certain that the actual cause for the disparity between the two is that Boeing is a publicly traded company while SpaceX is a privately held firm, which completely eliminates the need for management to focus on short-term profitability. When you look at stupid practices inside of a company, and cultural erosion, you can almost always trace it back to decisions that management is making to maximize profit. I'd go so far as to say, this is the ONLY reason why smaller companies are able to outmaneuver larger ones.

Reply on news.ycombinator.com