On Interviewing Programmers
thecobraeffect.blogspot.com
thecobraeffect.blogspot.com
I know it's weird, but I just ask them what they're learning, if they have personal projects and just use that to see if they have experience writing software.
Some people are super-excited about what they're learning, want to talk about the technology and end up making a good impression. Some people are only able to talk about what they're doing at work, the technical challenges and usually make a good impression as well.
Whiteboarding is usually kept to high level concepts like normalization, data flow, logical architecture and things like that.
I used to interview people by looking at their resume, and starting asking curiosity questions about things they worked on, and going from there. I didn't have anything in particular I was looking for, just a vague sense from how they talked about it.
That's more complicated when organizations want to avoid the appearance of IP taint, so, it'd help if they had a side project/interest to talk about. But I'm not assuming everyone has time for that. Also, they might be slammed by work, and in project mental space they can't talk about, and temporarily skewing the impression they give about their curiosity about other things.
What's really unfortunate is all the formulaic, going-through-the-motions interviews these days. We used to make fun of the practice of hiring based on firmness of handshake, how well you dressed, and ridiculous standard interview behavioral questions that might as well be shibboleths for the people who prep for them. Now most young developers are told to prep for conveying culture fit, and answering particular kinds of CS101 coding questions/hazing, which seems to be much the same thing.
That said, though I think I can do better at finding great people than that, I still need to remember to add a big dose of humility, to the idea that I can get a sense of a person as an engineer, just by talking with them. I believe I can do a pretty good job of that, with some conscious uncertainty, and I'm self-aware enough not to simply hire people in my own self image. But of course I surely have more biases than I know, and there's a lot unknowns, and our practice is not science. So I should add in extra humility than I'd naturally have about this, and find the right balance between vision of what I think is needed, and being open to seeing things I didn't already see.
I also need to recognize that I'll have people coming to interview, having read the "Cracking the Interviews of FAANGs and All the Other Companies Who Act Like Stodgy Fortune 500 Hiring Large Numbers of Commodity Brogrammers" prep books, and probably seen that in some/most of their other interviews. :) And if someone shows up expecting the current usual rituals, I shouldn't hold it against them, but rather, it's my responsibility to communicate that I'd like to try to do this a bit differently.
One fear is that I'll explicitly tell people I'm looking to do this "genuine", and some will interpret that as an empty platitude or conceit, in interview prep book behavioral right answer terms. :)
>One fear is that I'll explicitly tell people I'm looking to do this "genuine", and some will interpret that as an empty platitude or conceit, in interview prep book behavioral right answer terms. :)
This seems like a good problem to think about. I don't have a solution, but I think it's a good thread to pull.
So we usually try to talk about what they're learning on their spare time, any personal projects, any cool technology they're looking to learn... then follow it up with what have you done so far to start on this, any small toy projects on this new technology.
It's a weird organization because aside from COVID-19, we've had very little turnover (most people are with us 3+ year with an average of 7 years) so we've kinda changed up hiring a little bit over time.
I live to work, and have had a bunch of open source involvement, and have had serious side interests, so those interviewer questions are to my advantage, when I'm the candidate employee/cofounder.
But a lot of candidates will have families that take up all their non-work time. (IME, some of them are among the best engineers.) And those candidates with families might've heard that many interviewers will discriminate against them for that, wanting people who can regularly put in 60hr+ weeks and surge to 80hr+ of grunt coding. So, I figure they might cringe at talking about how they're spending their "spare" time. I'm absolutely not going to discriminate against them, nor do I want them to fear that they might be discriminated against, or feel alienated because they keep getting questions like that in interviews.
And for the people who're pursuing cool technology through their employer, I'm really hoping it doesn't sound like they prioritized adding this year's unproven hot framework/buzzword to their resume, over the success/risk of an employer's project. :)
The only people who had serious issues were those who said they were not learning anything at their current and had no interest in learning anything new.
BTW, just to be clear where I'm coming from about families, I think people can focus successfully on both work and family, at the same time. (If they're working in an organization/projects that don't end up wasting so much engineer-time that they think they need people who can be typing 80hr/wk.)
Yes, and please, to all interviewers, tell candidates WTF they’ll be expected to do/talk about in the interview, in the greatest detail possible, as early as possible. If FAANG can basically publish study guides you can give more than “here are the people you’ll be talking to, and when.” What about? Will there be exercises. Leetcode? Easy, medium, hard? One of these or is it like half the day? You’ll want to chat about networking? JavaScript trivia most people don’t remember because they use patterns that avoid those parts of the language, and have for years?
Again, if FAANG can give the kinds of guidance they do then Bob’s Mobile Dev Agency can give candidates something about what they should expect without ruining the process.
Like, it doesn’t all need to be scripted but most employers seem to think it’s fun to make candidates come in with no idea whether they’ll be white boarding all day or sitting around singing Kumbaya in rounds
(And hash map are not even that difficult, but I got asked stuff much more complex than that...)
Also I personally really hate it when you are treated like some giant imposter when you don't remember some pet CS quiz question when you are 15+ years out of school, and for supposed "leadership" positions. Attitude is like "ho ho, trying to pull one over on us".
True anecdote time. A few months after starting my first full time job, I asked my boss why he chose me over the other candidates. "You had the driest handshake" was his answer.
In the man's defence, my first job was as a microbiology lab technician. His explanation was that a dry handshake was a good indication that a person would likely have a better aseptic technique, thus leading to fewer false positive results when performing the tests. Utter nonsense, of course.
I've come to the conclusion that nothing does, and that the place to think about scaling and repeatability is in training, not hiring.
Yes there is still a tiny possibly that it is just a talker, but such possibility seems much smaller than the possibility that it pass a coding interview on a whiteboard and then he is not that good.
The college project thing applies to the workspace as well; in a team or a project there's always going to be only a few or one that does the most and best work, and a lot of hangers-on.
That is actually a source of impostor syndrome for me, because if I look at my CV there's a lot of really interesting and varied stuff on there, but it was always me as part of a team. How much of what I contributed was original thought vs just copy / pasting someone else's work and changing some values?
That's said impostor syndrome speaking of course, I know I'm a competent developer and ATM I'm working solo on a project. Slowly.
Sorry, that's against company policy.
1) You want your experts to be spending their time solving problems, not interviewing people.
2) As the size of the organisation grows and it becomes a more attractive target for lawsuits (fair and otherwise) it becomes crucial to make all of your decisions in a defensible way. 'The tech lead doesn't like this candidate' will get you sued for discrimination if it's the only thing you can say when accused of prejudice.
While saying that, just because one is an expert does not mean they are an expert at identifying other experts. It is it a skill set in it's own right.
1. Asking them to talk to you as if they know you is implicitly asking them to reveal quite a lot about their personal life; a life which you, as a prospective employer, has absolutely no business poking your big nose into. Don’t ask about their personal life.
2. As “JMTQp8lwXL” pointed out in another reply, this also opens your selection up to selection bias to prefer your personal preferences, not what would be good for the role. This kind of thing is, unfortunately, rampant, and leads to enormous monocultures where everybody who works at a certain place seems to be the same person; they all dress the same, like the same things, talk the same, and look the same. This is the very opposite of diversity.
I’d rather hire for those skills plus code reading ability because I’ve honestly never used a single “algorithm” in my entire working life.
Employers seem to be entirely ignorant of the power relationship in job interviews; interviewers all want to talk to the candidates like a peer, even though this is not at all appropriate.
And of course diversity for diversitys sake is the highest principle there is.
Also, all people with similar interests are carbon copies of one another.
As you sarcastically pointed out, diversity for diversity sake is a false virtue IMO.
The interview processes are still very fair compared to things like "performance" reviews and company "levels". People often pretend these types of internal processes are indicators of actual competence when they are usually just based on things like nepotism and corruption.
At least these algorithm interviews can successfully weed out people who can't write basic code. I can't say the same for promotion processes.
"Nepotism and corruption" is a very odd claim.
Does this mean that only family members and friends of managers get promoted? Or that you have to bribe folks in the company to advance? What do you actually mean?
What I would say is that most companies haven't really put much thought into building career ladders for their people.
Or into defining roles that provide enough breadth to capture a diverse skillset, enough autonomy to get the job done, and enough in the way of pragmatic constraints to scope the role.
Algorithms interviews only tell me that somebody can pass an algorithms interview -- one in which the solution space is very narrow, and there are a few correct answers.
In real-world engineering, this is very rarely the case.
There are always trade-offs to make. Good engineers know how to do things the cheap way now, and pragmatically invest more as we need to -- because they follow practices focused on, second only to delivering value, making the code easy to change.
In many cases I've seen people being promoted due to being close friends with the managers while contributing little to nothing of value. Also dishonestly claiming credit for others work is a great way to advance. Donald trump types get promoted.
> What I would say is that most companies haven't really put much thought into building career ladders for their people.
These career ladder guidelines often make the problem worse. They allow poor engineers to get away with avoiding real work in favor of soft "cross-team impact". Often the lower level engineers working on the difficult technical problems are making the real contributions. Good engineers should be focused on solving the most difficult problems, whether cross-team or not. They should not be focused on gaming silly HR guidelines, people who take these guidelines seriously make the worst engineers.
> Algorithms interviews only tell me that somebody can pass an algorithms interview -- one in which the solution space is very narrow, and there are a few correct answers.
> In real-world engineering, this is very rarely the case.
I don't agree. Psychometrics shows general cognitive ability is very important, and algorithms interviews to an extent seem like a decent test of it. Coming up with difficult solutions isn't happening without proper grasp of the basics and strong general cognitive ability.
But take my thoughts in this with a grain of salt: my bias is toward believing that most engineers place way too much value on coding work and not nearly enough on the (in my opinion) harder work of figuring out how to successfully execute an ambiguously defined project.
As the company gets bigger, the chain of trust weakens. The only way to keep something that resembles trust is to comeup with a process that everyone can trust rather than just trust each other. If such a process didn't exist it would always be one person's opinion vs someone else and there would never be consensus. This is made worse by companies hiring way too aggressively chasing that next funding round or IPO than hiring when they really need to. And would rather hire someone vetted by a text book process than trust their colleagues to look for diverse skills for their teams. This vicious cycle has been going on for way too long and will have to end sometime.
I feel strongly about this topic too that I wrote about sometime back, https://news.ycombinator.com/item?id=23777149
I have found good succes with the following approaches. Being in a small firm helped, but I haven't had too many difficulties following this in a large-ish firm too. (product unit in an Indian firm)
- don't hire in urgency. I believe in building a talent roadmap/planfor the team, capabilities over next 6 months, and thus hire someone who I see fit enough to grow into that role
- have a questionnaire sent before, which has open-ended topics ( views and perspectives on containers, infosec, cloud, imperative vs declarative, microservices vs monoliths, their fitment (and otherwise) to use-cases - etc)
- talk about a recent tech challenge the person was able to tackle and resolve, try getting to extreme levels of detail (to understand whether the solutiion was a true fix or just a patchy workaround), the tradeoffs made while going that route etc
- try to do a systems design session, jointly. the problem statement closer to the actual line of work
- try to use a seemingly open ended objective (cost optimization, reduction in MTTR. increase MTBF, increase scalability etc) relevant to our line of work or current set of problems and see if the candidate can find a good set of questions to find their way towards a workable solution, etc.
- for simpler stuff, hiring interns and then grooming them into productive, trained members of teams is often a useful way
- responsibility doesn't end at offer rollout. setting their teammates for success till they are in the system is the leader's responsibility
However, when am on the other side (am the one who is answering, pursuing a job elsewhere), I have come to hate the find-the-algoninja ritual so much that have stopped taking up any that forces such a deal. Am better off retaining my sanity.
Conversely, if you're interviewing, signaling that you are highly interested in learning turns you into a strong candidate.
I learned this one by accident when I interviewed at a company I didn't care about
Relevant excerpt from a blog post I wrote [1]:
> I was wandering around the booths with my bag full of swag. My eyes caught a pile of rubik’s cubes being given away by some company I’d never heard of. I wanted one! Of course I couldn’t just go up and ask for it directly, so I went and chatted with the guy manning the booth. His name was Vince. A few minutes later I walked away with my prized rubik’s cube in hand. That evening Vince called me up offering to conduct a full interview loop on campus. I already had a job offer from a company I liked, but I thought “Sure, why not? I could use the experience”
> I had no intention of joining the company, they were some boring finance business. There was nothing to lose. So during the interview I felt free to ask any question I wanted. When I thought I got the answer to an interview question wrong, I’d ask “I don’t think I did so great here. What’s the right answer to this problem?” (I wanted to learn the answer for future interviews!). When asked a challenging question I could grin and delight in the problem solving aspect instead of worrying about how badly it would reflect on me. (Remember my smart aleck remark in the intro? That was this interview.)
> Turns out they liked that: the next day I had a second job offer. At a significantly higher salary than my first one. Oh, and they wanted to fly me to New York in two weeks for an introduction to the company. I still didn’t want to join, but a free trip to New York? Sign me up! The company’s name: Bloomberg
> Bloomberg was a practiced hand at recruiting. They had a full two days of events to leave us starry eyed about the company (I came thiiiiis close to accepting their offer). While there, Vince told me I’d made a great impression by fearlessly asking questions, even when I was stumped.
> Since then I’ve stopped hesitating before asking any question in an interview. Let your curiosity run free! Don’t cage it! And as an interviewer, I can attest: sincere interest is always a good sign.
[1] https://zainrizvi.io/blog/the-interview-advice-no-one-gives-...
There've been various studies done which show that individuals who are under pressure to perform any creative work actually have worse output in the same amount of time compared to people who're doing that exact same creative work for fun
I think there's a famous study about people being asked to attach a candle to a wall with a box of nails that's the most well known example of this phenomenon.
So even if what you say is true, it's in your best interest to forget that fact and focus on the fun
And yeah, it was totally branded all over with the company logo and other symbols I didn't understand until after attending the open house
For more junior candidates add blantant defects or bad design into the sample code.
I have experienced this both as a candidate and as an interviewer. As a candidate, I found this a lot less adversarial and less stressful. As an interviewer, IMHO this gives a lot of insight into how a candidate thinks, without being a huge time sink, like 4 hour whiteboard tests or take-home assignments.
One of the best interviewing experiences that I had was when I was asked to bring in my laptop with some of my code to discuss. I had a side project that I was working on at the time so that was not a problem. I showed the interviewer, he asked questions, asked me to add a simple feature, I did. No algorithimc nonsense (which is basically about as reliable as a coin toss as to whether I will get the answer on the spot). No hours of my spare time wasted on take home challenges. It felt like less pressure than usual as it was code that I knew.
Either because we actually did real work -- the interview was literally "work with us for a day and see how you like it" -- or because the interview process primarily centered around exploring problems in a wide solution space, collaboratively with the interviewers.
For it to work, you obviously need to impose some constraints - it's unreasonable (IMO) to demand too much time and energy out of candidates, but it should also be complex enough to highlight a wide variety of skills.
But what's more important, is how you use it. It's an excellent tool in interviews, because you can pretty much spend an entire (technical) interview just discussing their implementation, and have more low-stress discussions around their choices and strategies.
A good and experienced candidate will both showcase good code, and will be able to discuss quite fluidly and in-depth around their various choices. And if you want them to expand a bit one some technical things, you can just ask them then and there.
But, again, I understand this is a wildly polarized method. Some coders would rather spend xxx hours on leetcode / hacker rank / etc. and learn every problem under the sun, and then tackle some random whiteboard question.
I guess if you're fresh out of academia, that's the method which most resembles your previous experiences with testing. Some are good test-takers and love trivia.
But for me, I'll stick with the take-home problems for now. Both when looking for devs, and when looking for jobs myself. Simply put, you get more data out of such problems. If a candidate cheats by hiring someone to do their work, it'll more likely than not still show up under the interview and questioning.
edit: But as mentioned, it all boils down to how well you design such problems, how they are executed, and how you choose to use them in interviews. There are many ways to effectively render them useless, or even hurt your chances at attracting good devs.
One thing that I don't see much in the discussion is the assumption that we all have time to go through the interview-course material which often is irrelevant to our day to day work -and also irrelevant to the advertised role.
But then again, everyone knows it, interviewing is just another role-playing. You do your part you get the role. That or you get in the company via a good recommendation avoiding all that sh!t.
Like someone in valley put it, "competition is for losers". It applies to jobs too IMO.
The hard fact of SE job search is that with most modern companies, you're going to get some sort of technical questions.
For ME, I'll much rather be asked questions about something I've just written, rather than random DS & Algo questions.
In fact, how many experienced know DS&A down to the bone, at the levels of Leetcode medium and hard, and can just wing an interview, with no prep? I'd imagine under 10%.
So yeah, I'd rather take a couple of hours to make some simple application, and review that, that to spend N times that combing through competitive programming questions.
(Most of all, I'd rather not have any of those, and let my previous work do the talking, but that's the world we live in.)
I think the author didn't lead msny interviews and only imagines themselves as the applicant. I had so many people apply for programming positions who couldn't even type the most basic for loop. The reality is, you need to find out the "level" of the applicant - are they a complete beginner, can they contribute basics, can they design medium sized projects themselves or can they be solely responsible for a larger one. That quoted approach doesn't help with that at all because many people will just tell you what you want to hear. Live coding and whiteboarding can give you good insights.
Every method of hiring candidates discussed seems to be accompanied by related horror stories.
And watch what questions they ask.