Hacker News Reader

Comment by gkoberger | Hacker News Reader

I ask everyone to bring their own project to work on!

It could be adding a feature to a side project, starting something new, or contributing to open source. (Or, we have some stock ideas if people need inspiration.) Rather than asking them to solve a problem they had never heard of before or use a codebase they're not familiar with, I get so much more out of watching them work in their own environment.

I can ask them questions about why they're doing something, and they tend to have much more detailed answers because they've been thinking about it for weeks. They're solving a problem they care about in a codebase they know, which mimics how working with them will be a few months in.

We can talk about tradeoffs and design decision, and I get a real sense for how they think. Plus, I've found most people are so much more comfortable and excited.

(If you're interested, I wrote a blog post about all the ways I've designed how we interview, to give everyone the chance to show off the best version of themselves: https://blog.readme.com/designing-a-candidate-focused-interv... And of course, I'm hiring!)

View on HN

Plenty of great programmers have no side projects. They have family or hobbies or something else that occupies their time outside of work.

Yup! We have projects and ideas ready to go if they don’t have one :)

For people like this, though, I’ve seen many use it as an opportunity to work on something they’ve been wanting to for a while and never had the time! A Hello World in a new language, merging in PRs on an old repo, writing a script to automate something or even a take home coding exam for another company.

In the example interview schedule in your blog post there's only an hour allocated for BYO project discussion. Is that representative of your typical interview schedule? The scope of some of your examples in the blog post may be challenging to cover in much detail in the first hour working with brand new collaborators, although perhaps is enough for calibration. Are you able to share any examples of the fun projects/ideas you have as suggestions for candidates that don't have a project of their own?

Yup, we do an hour. After that, it feels like there's diminishing returns. There's absolutely no expectation anyone will "finish", and we make that clear. It's the journey we're interested in, not the destination.

One that we tend to do a lot is creating an image gallery using Flickr or Unsplashed's API. It's vague enough that people can take it in whatever direction they want, and simple enough that you can get pretty far in an hour.

I was surprised it was that long, I can talk about a project for much longer than an hour but the average interviewer won't give a shit past 15-30 minutes.

It's not just talking, you'd also be adding a feature or fixing a bug or something like that :)

Could expand on that? I only have 45 minutes and can't have them bring in a project unfortunately.

Do you have some legal/security issues?

I ask people to send me code they've written via email or links to projects and go through that.

Sure but the worst case scenario is equivalent to a take home exercise, which many companies are happy to have you do. I have been asked to bring in my own work (I did have some) and it was the most pleasant interview experience I have had.

I see it as one of those "strong signal if it's there" things. If someone has loads of stuff to show you and they clearly love programming because they're doing it all the time, I feel like they are gonna be a great programmer. But of course if you're not in a position to show stuff, that doesn't mean you're bad.

My guess is most people are covered by some sort of IP clause for their main work, and the most visible OSS projects are still not run by very many people, so most devs are not gonna be able to show you much.

Yup, but it’s just plain dumb to ignore the ones that do.

I have a gigantic open-source portfolio[0]. I also go to great lengths to make sure that I’m a “known quantity” (like posting frequent exposition on fora like this one). I don’t feel that I have anything to hide. Everyone that has worked with me has found the experience enormously fulfilling.

It’s been quite jarring to see this ignored and treated with disbelief. That’s pretty much exactly the opposite reaction from the one that I would have, but I guess I’m out of touch with the way things are done, these days. I even talked to someone who told me about a friend that had invented some kind of machine learning candidate evaluation system, and relied on that, as opposed to human vetting.

One of my goals is to discourage people that wouldn’t want to work with me. I was a hiring manager for a fairly high-functioning, high-stakes team, and am quite aware of the value of making good hiring choices, which go well beyond simple expertise. Team dynamics are also very important.

I feel that getting a job as a result of misrepresentation is a big mistake, and a recipe for misery. So I don't hide my age, don’t pad my résumé, and make every effort to be as open as possible. If you don't want to work with me; fine. Let's not waste each others' time.

I have always sought to make it easy for people to figure me out. Since I live a fairly good life, and know my way around the playground, I feel that I'm a pretty good bet, but I have no problems if others think otherwise.

One of my favorite Twain quotes is "Let us so live that when we come to die, even the undertaker will be sorry."

[0] https://stackoverflow.com/story/chrismarshall

could you elaborate about the machine learning candidate evaluation system, if you remember? Interested to know the approach

It was hearsay. I know who the person that claimed it is, but I feel as if I might be irresponsible to say more than what I have. It's deliberately vague.

That said, I suspect that many companies have been exploring this. It's pretty obvious that many places are already using AS (Artificial Stupidity) systems to triage applicants.

I don’t feel that I have anything to hide. Everyone that has worked with me has found the experience enormously fulfilling.

I think I just threw up in my mouth a little.

Then you wouldn't like working with me. Now you know. Problem solved.

If you also act like a cock in person then definitely not. Even the most brilliant people in the world usually have the self awareness to not give themselves glowing reviews, perhaps you're great but have a social blind spot in that regard. If you consistently speak like that in person then many people would not like working with you and that would call your glowing self review into question.

Literally everyone? Ok :)

I have no idea why you wrote that comment. You do realize that this isn't Reddit, right?

This is basically a sub reddit. I was just responding to you like I would in conversation, not sure what the medium has to do with it.

Well, for me, you'll notice that I deliberately don't hide who I am. I deleted my last anonymous account many moons ago.

As such, I won't post stuff like you just did. It's uncouth and unprofessional. I consider this a professional environment. If you choose to look at it as a personal entertainment medium, then knock yourself out.

I'm a reformed troll. I used to be a right bastard. A great deal of my approach, these days, is an act of atonement. In fact, a great deal of my personal philosophy is based upon becoming a much more productive member of society, than I have been in the past.

I'm every bit as cynical as the next chap; more so, I'll bet. I regularly deal with some truly scary people, in my personal work on NPO stuff. I doubt there's a thing that you could say that would shock me.

It's also quite possible for very good people to exist in this world. I try to be one, myself. I know of many, many others. They are my heroes, and my models.

Please don't make the error of mistaking optimism, kindness, or politeness for weakness or stupidity. It's my experience that this can lead to...costly mistakes.

Please don't make the error of mistaking optimism, kindness, or politeness for weakness or stupidity.

My post had nothing to do with any of this.

All I was pointing out was that being excessively boastful and claiming that everyone, without exception, finds working with you "enormously fulfilling" is just unbecoming. I feel like many will find that distasteful and you should leave other people to rate their interactions with you. I doubt there's a single person in the world who has a 100% success rate with regards to positive social/professional experiences with others...

I could have definitely worded it more politely and honestly if this account was tied to my personal name I probably would have worded it more politely, so you've got me there :)

All that said, you do have a point, and I'll refrain from saying that I'm a kind and openminded person that people tend to (as opposed to always) enjoy working with.

I posted incorrectly. It is not 100% universal, but I'll bet it's a lot more than you might think. I've spent the last 40 years, getting along with some of the most difficult people on Earth. In fact, many of them are ones with a history of violence, and it's always a good idea to be polite in that kind of company.

Geeks are easy, after that.

How do you compensate for the fact that you don't know the codebase that well? I feel like a lot of the value of interviews is that you see how a candidate thinks through a problem that you have also thought deeply about, which gives more fruitful discussions.

This is a great question. Guess OPs approach only works if they are hiring someone much more junior than themselves so that experience trumps code/domain familiarity.

Not at all! I’m pretty experienced, but most people I interview or hire are much better engineers than me!

Imagine you’re taking a class on a topic you don’t know. You can tell pretty quickly from how the instructor talks and explains something if they know what they’re talking about. It’s very similar! I’ve certainly interviewed people much more skilled than me in languages I’ve never used, and still got a pretty great sense of their expertise.

Well, that's not how I tend to manage day-to-day. I don't give engineers I manage a problem I've already figured out and solved, and then expect them to do it the way I would.

Rather, the engineers on the team are usually building something new, and figuring out how to accomplish it is their responsibility. This way of interviewing mimics that a lot closer.

If someone can't explain to me a project they're working on, then they probably won't be able to do the same while we're working together. And that's just as important of a skill as writing code.

If I'm expecting interviewees (who are nervous!) to learn and understand my codebase/problem, why can't I do the same for theirs?

Love this attitude. It's a great attitude and it shows. A very common thing I notice with interviewers is that they do one interview over and over again until it's super polished, but paradoxically, they won't compensate for that and in effect, the bar gets higher and higher for later candidates than earlier ones. Flipping the script not only is a great way to balance things here, but it also says a lot about your own confidence. I might have to lift your approach here!

Yup, exactly! The interviewer gets annoyed, since it's "so obvious" how to solve the problem... much like anything becomes obvious if you've done it a few dozen times.

Or they tune out. Or they dismiss any variation as "wrong" rather than giving it a chance.

I also love this attitude but I disagree about this negative effect you're talking about in reusing problems.

What happens in practice is the interviewer compares candidates answers to each other, not to his/her own way of solving it. It doesn't matter what whether the solution is "obvious" to the interviewer. If I hired someone who only did OK on this question and they ended up being a great engineer (and that happened multiple times) that lowers the bar for the test (or tells me it's not a very good test etc)

You're missing the part where the interviewer is changing with each interview (and the rest of their life in general). They phrase the problem differently, get anxious as the interviewee approaches different parts of the task. So, the interviewer thinks they're giving the same test and can fairly compare the outcomes. The GP's point is that's just not true

Whether it works out to be a harder or easier interview over time because of these changes probably depends on the individual interviewer.

Sorry but the pointed out problem is both way more commoin and way way more of an issue. Interviewers talk about "I just want to see how someone thinks" but the implicit biases and the overuse of problems leads to unconscious expectation setting and rigidity in the interviewers. In fact, I explicitly test for this by giving valid answers using e.g., techniques that I don't think the interviewer is familiar with precisely to see how open minded the interviewers actually are. Their response is amazingly revealing about the interviewer and the organization behind them.

> What happens in practice is the interviewer compares candidates answers to each other, not to his/her own way of solving it.

I think that's _ideally_ what happens in practice. But as sibling comments mention, is that what happens in practice?

Interview fatigue is very real. Many interviewers are not positively incentivized to put in the substantial amount of effort to conduct a quality interview. It's something that is simply thrown on top of their usual workload. Come promotion time, there's no reward for doing it well. But, there is punishment for noticeably "screwing it up". Often, that means that interviewers put in the bare minimum effort with interviewing (as with other tasks) to not get punished.

I've seen competent leaders avoid this in the past by emphasizing that interviewing is the most highly leveraged thing you can do for your organization and your own career, because you are step-function increasing the effectiveness and capability of your own team if you do it right. But in order for that to happen, there needs to be a real culture of working as a team and not just a disembodied group of ICs: a culture which rewards increasing the effectiveness of the team (instead of merely focusing on ones own tasks) and doesn't just pay lip service to it. It's tricky. But I think the subthread GP's approach is a clever and creative kind of a lateral thinking that shows how to do just that.

So practically - do you or some of your team get added to their project so you can track what they're doing?

do you (or others on team) ever make suggestions on how the code should be written? I think this would be an interesting way to pre-teach the hire on how you like things structured.

on edit - I guess since you ask them questions that implies making suggestions as well, but just wondering how it works.

Lots of why did you, why didn't you, what about this, how did you deal with that (and back to why). :-)

I signal a lot of things like testing by asking them. It gives room for the candidate to ask those sorts of questions back about how we do things.

It's all done live! The first time we hear about what they're working on is during the interview, and we never keep any of the code. It's great to see how they can articulate what they're working on, combined with actually working on it.

We sometimes might make suggestions, but rarely for the sake of code quality. We'll once in a while ask for something unique just to get them out of a rut, and see if they can improvise. Otherwise, we're here to learn and listen, not teach :)

Ask them to explain it to you. The quality of the understanding that you gain, and their willingness and ability to engage with your curiosity, is one measure of the candidate. A coarse-grained qualitative measure perhaps, but certainly one aligned to dev team productivity.

How do you maintain consistency across candidates with this approach? One of the struggles I've seen with interviewing is making sure that candidates are judged on a fair and consistent scale across interviews, candidates, interviewers, etc. Do you find having a different project for each interview makes getting a consistent and comparable signal out of interviews challenging?

Good question! We’re lucky we don’t need to worry about this for engineers. We never choose between two candidate, rather each candidate is judged on if we want to work with them or not.

It’s an advantage we have as a small company! We can rate people based on whatever criteria gives them the best chance, and we can find a ton of great candidates that maybe wouldn’t make it through a traditional hiring interview at a larger company.

Then your method doesn't scale.

I think it does. But even if not, that's fine! Until we have to scale it, it's an unfair advantage we have.

Like YC said, "Do things that don't scale."

http://paulgraham.com/ds.html

What makes you believe that? Do you believe that the e.g., algo approaches create a strictly rank orderable list of candidates which actually correlates to their on-the-job impact? Also, why do you think such a thing matters more at scale than when a team/company is small?

From the op - "We never choose between two candidate, rather each candidate is judged on if we want to work with them or not. It’s an advantage we have as a small company! "

That doesn't answer my question about why you claim it doesn't scale.

When the company gets big / popular enough, they have to choose between 2 candidates for the same position. All these policies will then go away.

Hey thanks for the reply. I don’t really understand this answer to be honest. How do you compare fairly between two or more candidates applying for the same position with this method? If you’re judging all applicants on the “would you want to work with them scale” what are the common parts of the scale across applicants as it applies to this technical project? How do you ensure the judgement of wanting to work with someone is fair within a pool of applicants for a position?

Given all of the vagaries in the hiring process, do you believe that e.g., algo approaches are actually fair in the way that you intend?

Hmm, I think we're just looking at it differently. Why does fair mean "everyone is graded exactly the same, whether they're good at this certain type of problem or not"? Couldn't it be argued that the most fair way to judge people is to treat everyone individually?

A lot of engineers don't work on their side projects because they can't. Every company I've interviewed with requires IP assignment agreement. From my experience, it's standard outside of California where they're banned.

If you asked me to work on something, it would be a toy project. Because I can't contribute to open source and if I build anything substantial it will be taken from me.

You shouldn't punish people in this situation. It's extremely common, maybe even the majority of coders outside California.

I already answered this elsewhere: https://news.ycombinator.com/item?id=24457320

That being said, I've interviewed hundreds of candidates over the past 5 years, and not a single one has cited IP assignment as a hesitation.

>Every company I've interviewed with requires IP assignment agreement.

of course they do, you're a knowledge worker for them. they need to own what you create.

a standard phrasing includes "related to the employer's business". see for yourself: https://www.lawinsider.com/clause/intellectual-property-assi...

"during the period of his employment relating to the business of the Company". If your contract doesn't have the words "relating to the business of the Company" then add those words in yourself, signing and dating the change and telling the company of your change.

you're not their slave you know. they don't get stuff you make in your free time. however, not replicating the company's business is a fair requirement, since they might share enough information with you to do just that. I think it's fair for them to get everything related to their business.

Your candidate-focused interview process is very inspiring! I really like the "bring your own project" idea and it also looks like you've cleverly integrated your product into the interview experience (I'm assuming that is what powers the personalised interview website). I also think the general tone and styling of your content is very considered and engaging. My honest first impression is surprise that I haven't come across your brand yet, but I'm definitely going to learn more.

That's an interesting interview approach - I really like it. Do you find the lack of context (about the interviewees project) on the part of the interviewer to ever be an issue?

That’s the best part! You get to see how well they are able to articulate what, how and why they’re building. That’s such a huge part of working with someone, which you don’t get if you tell them exactly what to do.

Wow. Best new interviewing idea I have heard in a long time!

This is well intentioned, but have you had any of the applicants try to game it?

Sure, people prepare! But that's a good thing. We want everyone to do well.

If we sense they're doing something they've rehearsed a bit too much, we might push them to do it a bit differently. "What if you tried to do this by _____", just to make sure they can think on their feet.

Honestly, though, the best candidates for us are the ones who wouldn't try to game it. They hear the prompt, and are just so excited to show and talk about a project they're building that it's not a problem!

I had a great conversation with one of my cofounders about this. First off, if you're only asking questions to a point where they are gameable then your questions aren't very good. The example I was trying to explain to him was that I look for 'aggressive curiosity' and wasn't sure I wanted to put that into the job description. He laughed and asked how far could someone game that as we keep asking follow up questions on any aspect of the discussion?

Yes! "Curiosity" is the #1 thing I look for, too! And this question is great at it.

It's a great proxy for empathy and understanding. If you're curious, it means you want to learn the why behind why other people do things.

Indeed. And true humility.

"Less certainty, more inquiry." --Erik Seidel

Interesting. Does anyone ever fail this part of the interview?

Can't speak for the GP but IME, absolutely. It's much easier to dig in deeper with (much) clearer signal about somebody when they are talking/working with their own code/project. Less distractions/confounding factors/proxies, more direct inquiry and you can go as deep and broad as you want to map the edges/gaps of not only what they know but what they care about/prioritize and why. The proof is in the code.

How do you avoid bias? It seems that if a personal project aligns with the interviewer's preferences or tastes it could create a huge bias.

Is there any more bias than the interviewer picking the question to begin with?

By letting the interviewee pick, we're letting them decide which of their "superpowers" to show off. If they can explain why they chose to do something the way they're doing it, and show they understand the tradeoffs, that's good enough for me.

There's a benefit to having a standard question that you can grade independently, you can uncover your own teams biases by having two people evaluate a candidate. One who met them, one who didn't. It can avoid interviewer bias with decision.

It sounds like you're putting that system of bias under the rug.

Reply on news.ycombinator.com