They provide the candidate with a piece of terrible code, ask for a code review of as many issues as the candidate can find, and also for a refactored copy of the code. Both of which are completed remotely on the dev's own time.
This is before even meeting the candidate in person. Once we meet in person for the first time, another terrible piece of code is provided -- in this case a basic data structure -- and another code review is requested. The candidate is given 15 minutes to complete it.
It's interesting what comes back. Some people comment only stylistic issues, others only logical or architectural issues. One person got up and left during their alone time.
Other than these, no coding is required.
So far, it's worked out pretty well. And when I was going through it, I enjoyed it.
Our take is more that if you can recognize what is wrong with the code on multiple levels then you can most likely write good code.
This is easier to check on non-trivial code when done through a code review then by getting the interviewee to write the same kind of code that every other interview looks for. The submitted code sample does this as well.
I'd also want people on my team to be able to successfully refactor legacy code just by looking at it and being able to identify where things can be better cleaned up and tested.
So that's not my issue. I was pointing out that your interview process seems to involve only code review, based on your comment above. Put yourself in an interviewer's shoes: in two different instances, you've been presented with bad code and asked to code review it. Personally, I would be concerned that (1) your company writes bad code, and (2) I'm going to be expected to spend all my time reviewing and fixing other people's bad code instead of writing code myself.
Now I doubt that's actually the reality of working at your company (at least I hope so!), but first impressions are very important, and that's what your interview process would tell me if I were interviewing there.
It's not a first line, though. We tend to start by actively recruiting/headhunting. We cherish recommendations from current or past employees, but we'll also open the position to all on the off chance we get a good application through traditional routes. HR only becomes involved with the process after an offer has been accepted or if the hiring manager has questions/concerns.
After an informal lunch with the hiring manager, you'll be invited to meet with some senior developers for about an hour. These meetings are mostly to figure out what the candidate thinks their strengths and weaknesses are and gives us a chance to talk about the stuff we plan to work on in the near and not-so-near future.
Once we have a fairly basic understanding of each other, we'll come up with a project. Think your strength is on the front end? Here's an API we wrote. Build a widget with the data. Think you're strongest at databases? Great, here's a database project.
We don't care if you have a lot of questions or if you fill in the blanks. We give as little direction as possible at first on purpose. Write it in whatever language you want to write it in. We want to see how the candidate works. If they ask questions, great. If they don't, great. Do they ship something?
We make sure to stress there's no wrong answers. At all. The projects are designed around your perceived strengths because we'll figure out your weaknesses and will compensate for those during the 90-day trial period all new hires have to go through anyway.
The point is to see something we can discuss in a later, final round. Even failure to complete is acceptable, so long as you can articulate why.
None of this is hard and fast. We have had candidates who, for whatever reason, can't or won't write code outside of their current position. We'll adjust. Our process isn't rigid at all. We try our best to take a holistic approach. It's only bit us in the ass twice, mostly due to culture issues because the hiring manager made his decisions unilaterally. Since then, all senior devs and a few others on the team meet all candidates before hire and can give management input.
Completely agree.
I'm still not sure what these "challenges" even prove. If you're a developer, you should have reams of examples of code, a github account with work, and plenty of other avenues for people to check if you can do basic coding from scratch. Even a junior developer will/should have several examples of work they've done while in school.
I've been in a lot of "technical" interviews and once they start asking coding questions, I usually ask them if they've looked at my online portfolio, or the examples on my github account. 95% of the time, I'm able to pull an example which deals directly with that they're asking.
Doing this has two effects. One, it proves I can do what they're asking. And two, it puts the responsibility back on the interviewer. It's like asking, "Did you do your due diligence, or are you wasting my time asking me this question?"