This is also the hardest to communicate as it is something inherent to the candidate. It's not personal, but it gets personal at this point.
Not my experience. At least when hiring for programming positions, the typical fatal issue is that the candidate's coding is weak.
In my first role as a hiring manager, I didn't stress coding tests for candidates with long work history on their resume. Since then, I learned better.
I've seen candidates with anything between 1 and 15 years of full time coding work on their resumes fail to answer basic fizz-buzz level coding questions.
At least for the candidates I've seen, the biggest single reason why they get rejected is failure to perform the one main function their job requires: coding.
I haven’t seen this. I have, however, seen lots of junior interviewers ask idiotic questions and wash candidates out for bad reasons, and then turn around and claim the candidate ”failed at fizzbuzz”, just to cover their own ass (or because they think TopCoder medium questions are “fizzbuzz”. Equally stupid.)
Me personally? I’ve never interviewed a senior person who couldn’t code at all. Not after the phone screen, anyway.
Why are "juniors" interviewing candidates? Let alone juniors asking "idiotic questions", and having decision-making capacity?
> then turn around and claim the candidate ”failed at fizzbuzz”, just to cover their own ass
Coding questions are among the most quantifiable and objective questions there are. If you know of something even more objective, let me know because I'd love to try it.
They are repeatable and have well-defined success criteria. Ideally you have two interviewers in the same room, and it's very clear if the candidate did or did not solve the problem. You can't really fudge it, even as a single interviewer, unless you're willing to outright lie that the candidate didn't write a working solution when he did.
> I’ve never interviewed a senior person who couldn’t code at all. Not after the phone screen, anyway.
Yes, if you do a coding phone screen, then obviously you're screening out those who can't code. I'm sure you saw some impressive resumes fail that screen, though.
I don’t know if you’ve noticed this, but the average age of programmers these days is just slightly above “right out of undergrad”.
If you wait for the senior people to interview everyone, you’re going to be waiting a while.
”Coding questions are among the most quantifiable and objective questions there are.”
Bullshit. Coding questions feel objective, and programmers love to pretend they’re objective, but like any other human evaluation process, they’re subjective. It’s not like you automatically get the job if you get the answer right.
Unless you’re verifying that your team is generating consistent, reproducible outcomes, I can pretty much guarantee that they’re not.
Still, interviewing is so important, that I typically give it a high priority. Even when I had only one other senior engineer, I made sure each candidate spent a long time with each of us.
Typically, I'd go in with a junior developer, and he'd go in with some other junior developer. I'd even see the same candidate twice rather than have two juniors interview him.
I've been on the other side of that kind of interview and I know how badly it can get messed up.
> Coding questions feel objective, and programmers love to pretend they’re objective, but like any other human evaluation process, they’re subjective.
I don't see a reason to swear, as this is a friendly discussion. Either way, we'll have to agree to disagree.
Coding questions ideally yield an answer that can compile and run and return correct output for at least some valid sets of input. That's about as objective as it gets.
Except for coding style (incl. naming convention), code organisation, test coverage, arch style, level of abstraction..plus a myriad of other things. I don't know anybody who treat the code in a coding question as a black box. It gets about as subjective as you can get when reviewing, with the (objectively) correct answer weighted the least.
Shit code that gets the right answer rates worse than code which ticks all the above subjective boxes but gets the answer wrong.
> Unless you’re verifying that your team is generating consistent, reproducible outcomes, I can pretty much guarantee that they’re not.
It seems as though there are a fair number of things like this, where people sort-of play act science but don’t bother with the rigor. I guess it’s a type of cargo culting.
But I see no other way to test if the person is competent than sitting them down with a problem to solve -- if only to have them explain what they do and don't understand, and what their process is.
My goal is to figure out the candidate can do their job - write software - and that myself and the rest of my team would enjoy working closely with them on their daily tasks.
How else do I figure that out, if not by a coding exercise?
Programmers today think that TopCoder questions are “FizzBuzz” tests. That’s idiotic. Joel Sposky wrote that blog post to say that you should do an absolutely minimal check that someone can write code, then move on to more important things. It’s just bizarre how people have twisted that over the years.
But to answer your question: focus on what matters. Communication, writing skill, the ability to read code (this is easily 10x more important than algorithmic wizardry IRL), good habits (e.g. “does this person insist on putting all of their code in a single file?” - an actual thing I have seen!!), opinions that match your team, etc.
All of these things can be answered in interviews, but nobody cares to try.
> the ability to read code
How do you test for that in isolation?
There are exercises that do that: you show the candidate a piece of code, and ask them to find the bug. They have all the worst issues that folks here are complaining about, amplified: stressful, awkward, etc. In my experience these types of exercises are also more random: i.e. depend more on luck. Sometimes great candidates fail them, and poor candidates get them right on a fluke. So they are overall less indicative.
Ultimately, though, it's back to the first point: how do I test if the candidate can do their job, i.e. code?
A candidate may communicate really well, say all the right things in best-practice questions like the one you mentioned, write well in English, but still unable to code their way out of a matchbox.
”In my experience these types of exercises are also more random”
Well, “your experience” also says that you regularly encounter senior candidates who can’t code at all. I doubt the statistical validity of your experience.
Interesting, and I won't look up the article again before writing this comment.
I remember it as saying he was having a hard time finding people that can code at all, that a FizzBuzz is weeding out a large percentage of applicants. I don't recall it as a gateway to move on after.
Furthermore coding questions stop being quantifiable and objective as soon as you add in, "What feedback do you give, when, and how?" Two interviewers asking the same question of equivalent candidates can get very different results based on how stressed they make the candidate, what hints they offer, what followup questions they ask, how well they telegraph whether the candidate is on the right path, and so on.
Sure, but these are highly correlated. If you've been working in startups for a few years, you will have some experience in both coding and interviewing coders.
Even most of the successful large companies are pretty good at cultivating interviewing skills among their employees.
> Furthermore coding questions stop being quantifiable and objective as soon as you add in, "What feedback do you give, when, and how?"
I would moderate that statement as "become less quantifiable and objective". They're still more objective than "tell me about the project you are most proud of" or (worst) "what is your biggest weakness?" (perfectionism, duh!).
You can also control many of these factors by standardizing interviewer behavior. For example, "no feedback of any kind until the candidate writes a working solution", or working off a predefined set of hints.
Even most of the successful large companies are pretty good at cultivating interviewing skills among their employees.
I would put it the other way around.
My experience is that most startups just put you in front of people and tell you to figure it out. People who have done that for a bit wind up with really bad interview habits because they never get corrected. My experience with large companies (Google and Amazon) has them giving interview training. I learned a lot more about interviewing from that than the informal practice that I got in startups.
Some do, sure. And that's the difference between a startup and a regular company right there: that "figure it out" bit.
You'll mess up and fumble and come up with crazy ideas, 95% of which are worse than what the established companies are doing.
But you'll learn a ton along the way.
It's messy and inefficient but that's how startups are.
I personally learned a ton from hacking interviews at startups and all that "figuring it out".
Also, guess what? All these sacred "FAANG interviews" that seem to have come down from the sky, etched on tablets? They were developed by Microsoft back when it was a startup, then evolved by various SV companies at their startup stage, too.
The next great idea in interviewing also won't come from the thousands of engineers obediently marching to the tune of the same principles everyone is using, but from some startup trying something crazy and ambitious and miraculously getting it to work.
Only in the most trivial cases (for statements, not necessarily solution). If you want to assess engineering ability, specific coding questions are a small, tiny sample of the space, and generally the more difficult the problem the less useful it is.
Not sure how common this is, but I've been one of those flame-outs. It was during a Google phone-screen.
I was given a very simple programming problem, and something in my brain just broke (figuratively).
In retrospect it was probably from being too tense because I was star-struck about the employer.
But on that one particular interview, I could barely have written printf("fizbuzz\n");
For example, if I test-run a new question, and it takes my engineers 20 minutes to solve it, I will add 30-50% handicap for stress. I.E. if a candidate manages to solve it within 30 minutes, that's roughly equivalent.
Unfortunately, there's no way to account for someone having a complete blackout such as you describe.
I am doing just that.
> But literal whiteboard tests are a recipe for complete blackouts.
Not always. I've done a lot of whiteboard interviews too. It's a different way to discuss a problem, and one that people do use at work.
Most design sessions happen with a colleague or two in front of a whiteboard, not sitting at a workstation.
I disagree that blacking out is so common, though I understand it does happen occasionally. Also, we typically do at least 5 interviews in every onsite, one blackout wouldn't destroy the chances of an otherwise strong candidate.
Fortunately I don't find myself in that style of interview anymore. The guy interviewing me already knows I can code, so all we are really trying to see if there is a culture fit. My preferred interview location is over beer.
My comment referred specifically to candidates with at least one year of full-time hands-on development experience under their belt. The typical candidate I interviewed had 3+ years of experience.
If you want to code, then you should have ample opportunity to do so during a year or more of full-time employment.
If you were working in a full-time programming position for over 3 years yet you can't write even a trivial amount of working code during the interview, then clearly you either lack the capacity or the will to develop your programming skills.
Programming nowadays is easy - everyone has a powerful laptop and internet access. It's not like you need privileged access to some $100k time sharing machine using punchcards, and learn machine-language from obscure mail-ordered manuals.
What excuse does anyone claiming to "want to code" have for not coding already?
I'd be quite suspicious of anyone waxing poetic about their genuine heartfelt love for coding who somehow couldn't demonstrate any coding ability.
Would you believe someone who swears they "love long-distance running", but collapses after 50 yards and says they just didn't have the opportunity to put this great love to practice?
Genuine question... What kind of question(s) would I ask to get someone to demonstrate they can code? If my questions are effective, I'm fine. But if my questions stink, it'd be easy to overlook that fact that many excellent people wouldn't be able to answer them well.
That's an exercise I do for hiring in general. For example, every time a hiring process change is proposed, I run it through my existing engineers, and make sure I would have hired them by this process.
For example, in a place that was starting to get a lot of great candidates, someone proposed tightening the education requirements of the preliminary screen. I quickly observed that would have disqualified one of my best performers, who didn't have a CS degree at all. The proposition was therefore struck down.
My challenge internalizing this phrase is that I have a hard time latching onto the idea of simple. It's a highly subjective measurement. For example, if I ask an to implement FizzBuzz & they can't do it, to my satisfaction, are they a not worth hiring? I suppose it depends on what satisfies me. If my criteria for hire/no-hire is whether they wrote concurrent & streaming FizzBuzz, maybe my expectations are unrealistic. If I'm expecting someone to write FizzBuzz in as few characters as possible, am I going to flunk anyone who implements a verbose concurrent & streaming FizzBuzz? Is there a case where it makes sense to hire someone who implements Enterprise FizzBuzz? I may struggle myself with keeping things simple, so there's that to consider :)
A single question is pulled from subjects vast enough to fill up years of college and there is no time to research, as one would in the real world. Binary pass/fail with your pride/livelyhood/family on the line.
Please state your last exam conducted under such conditions?
Oh and your questions probably stink. I've gotten plenty of them with recursion that might take ten minutes in seclusion but simply not possible under the gun.
There is a difference between me asking you how to optimize a graph algorithm (good luck--I probably couldn't do that on the fly with references in front of me) vs giving me a basic FizzBuzz.
If I let you pick your language for your FizzBuzz, you need to demonstrate at least some of the following things:
I/O of any flavor (I'll even accept Logging if you don't do standard I/O or console I/O)
String formatting
Modulus Operator
Nested-If statements (or equivalent--Matching, case/switch, etc.)
I'm not looking for production code. I'm looking for 10-20 lines of "A High School senior with a computer class could pass this."
This is not limited to programming, sadly.
I used to have to interview VLSI designers with "Draw me an inverter", and 70% of them couldn't pass.
And then some large percentage of "enterprise software development" is just hooking up buttons and text fields to a database, and doesn't require any more algorithmic complexity than a few if() statements.
And then some large percentage of the remaining work can be performed adequately by just picking some appropriate ready-made solution off Stack Overflow.
Put them all together and chances are, you can skate by for years (and actually perform acceptably) without ever actually needing to write FizzBuzz-level code for yourself.
Good/excellent programmers rarely need to interview except as a formality. Consequently, you are, by definition, interviewing weaker programmers.
2) Tool use masquerades as programming
Visual Basic 4/5/6 was one of the original sins in this genre, but it continues now with other tools and frameworks. Being able to point-and-click a tool and plumb it together isn't programming--but people will claim it is. Then when someone asks for an actual program--those people fail.
This does not necessarily mean that those people are not productive. However, it does mean that they can't actually function in a position where they are expected to program.
Indeed this is a fascinating question, and I explored it a few times.
Some folks who have multiple years of "software engineering" on their resume worked in a place where they did some form of "paint by numbers" programming, producing some work by modifying templates with limited (at best) understanding beyond the specific blanks they were filling.
An example is folks who work with large frameworks that are designed to be extended with little to no coding, such as WordPress and Drupal.
Then they want to advance to software engineering, so they recast their experience - of essentially configuring and deploying software, i.e. sysadmin - as "programming", and get those 5 years represented as "a full time programming position".
There's variants of that, basically "developers" doing very limited ant-work with large frameworks they don't understand and couldn't rewrite from scratch if their lives depended on it.
There's also a bunch of languages and frameworks invented for the explicit purpose that someone with little to no CS knowledge of programming talent could produce useful work.
Most of us here live in a bubble of startups where we often create products from scratch using lean (or no) frameworks. In reality, a scary amount of the "IT work" out there appears to be what I described above. Possibly a substantial majority.
Pick up a good introductory programming book. I recommend Think Python 2e:
http://greenteapress.com/wp/think-python-2e/
Go through it cover to cover and do all the exercises. All of them. Don't skip a single page or exercise, not even if you think "I already know this", and certainly not because it's too tough.
Once you're done, you should be able to start applying for jobs in whatever language that book taught you. If you're still not passing interviews, take on a serious - but manageable - programming project, such as some simple application for yourself or someone you know. If you're a web developer, a simple dynamic database-driven website is an excellent choice. If you want to do front-end, a simple Javascript game will teach you a lot.
You don't always have to apply for new jobs so early. Lots of tech shops have some teams where actual programming happens, even if most of the rest of the company is "paint by numbers" pseudo-programming. Often you can start to take on more responsibilities that involve actual coding if you show the necessary abilities. You won't have to change companies, or even change teams.
Good luck!
It's because current tech interviews are a complete failure and none will admit it. In ten years you'll hear a new fashion and today's will be decried a failure.
Now I certainly could do it in the course of a job within a week, with a day or two of research. But not under the gun.
That is the tech interview failure in a nutshell.
> Please state your last exam conducted under such conditions?
University entrance exams.
University graduation exams.
Any other test that you do which you need to pass to enter or continue your career.
Perhaps you should start listening.
> This isn't some high-level algorithm analysis on a whiteboard with a panel of examiners grilling you.
Happened to me just three months ago. The req is still open, wonder why?
To reiterate:
- Exams have a topic known in advance.
- Exams never have a person sitting there waiting for the answer.
- Exams are never 1 or 2 questions binary pass/fail with incredibly high stakes in unfamiliar territory.
Which is the brake pedal? If only.
(As for 'topic known in advance', I'd suggest that if you don't know what topics that you might be asked about in a job interview, maybe you shouldn't be interviewing for that job.)
We ask you to code a simple task that you should be able to do in your favorite programming language.
Not entirely sure how the discussion of coding interviews veered towards obscure knowledge interviews - the former was expressly designed to supersede and eliminate the latter as one of its core goals!
The job was for working on bare metal provisioning using yaml and Python, but I could 'use your favorite programming language'.
Before you think this was some third-rate body snatcher, it's a more well-known name that you've heard of.
They still ghosted me, after telling me what an 'amazing' job I did.
It’s centered around solving a reasonably common place problem, and doesn’t require anything fancy to solve. It’s more of a test in how they programmatically solve problems over anything else. If somebody struggles with it, then they haven’t been bamboozled by some acedemic algorithm problem, they’ve just shown that they’re not going to be able to write clean code unsupervised.
The opposite, shown they can’t write supervised under the gun, which never happens On the job.
I think I can do okay in coding interviews, but only if I prep for that style of work first. It's definitely different. I'm perfectly (or at least acceptably) competent in my day to day.
I don’t think it’s reasonable to say that people’s problem solving skills (which is what we’re actually testing) become exponentially less efficient during an interview. But even if it was, the test is still doing it’s job, because what would we expect to happen when we give a person like that a deadline, or ask them to review a PR?
> remarkably inefficient solutions
The first one usually is. As the problem gets clearer, better solutions become obvious. That takes relaxation and calm thought however, e.g. Archimedes in the bathtub.
> I don’t think…exponentially less efficient
Doesn't need to be a large difference. With multiple candidates, even 10% is decisive.
There's also the psychological factor of knowing something makes the "knower" think it's easy while the others believe it's hard.
I do lots of interviews as a contractor, and must say they have little to no bearing on my ability. In the real world I've solved every problem encountered, given some time. Interviews almost none, the few successes depend on whether I wrote the same algorithm in the last two weeks by chance.
A random throw of the dice at the craps table would be just as accurate and less stressful for all involved.
Likewise, many interview programming tasks can be are ones a competent programmer should be able to do with the tools of the trade accessible (namely Google and time to think).
Some simply are not. I was once expected to write, debug, and modify a concurrent chat server in an hour long interview. I do think that’s a reasonable programming task, but not one to be completed in an hour, under scrutiny. It’s the equivalent of being asked to rebuild a motor in an interview : definitely doable under real world conditions, but not a realistic test to determine whether to hire someone.
Like, how? I used to hire undergraduate student sysadmins / coders, and we had zero candidates fail fizz buzz. None! The worst we saw were people who maybe had an awkward solution to the problem, typically because of unfamiliarity with the modulus operator.
I should also stress (again) that back then I brought candidates directly to onsites when their recruiters sent an impressive resume with 3+ years of "full time programming experience".
I learned the hard way that most people who unknown recruiters are eager to push through your hiring funnel are unemployed for a reason.
This must vary from company to company, but it's clear that tech companies spent tons of time and efforts in technical screening. Most questions asked in these interviews are also technical. It's hard to see why candidates would even have a chance to talk about their political views
It was a good lesson for us, because it confirmed what we always kinda knew. Doesn’t matter if you’re good at writing code, we need to get along to do good work, and hiring somebody people can’t stand being around is a terrible idea.
I have a ~90% success rate in guessing whether I'll go on to the next stage / get an offer based on how warm the interview is to me within the first couple of minutes of the interview. The interviewers pre-judged you positively are rooting for you to succeed, and those who are biased against you are looking for reasons to not pass you. (This is assuming, of course, some very basic level of competency and that one passed the initial screening without lying)
Most people get rejected at the resume stage. Most who have acceptable resumes get rejected at the first screen. All of this is invisible to software engineers who only see candidates at the last interview step.