Out of curiosity, what sort of (non-personality) mistakes did you see?
1) Incorrect ordering of output "Buzz Fizz"
2) Printing only Fizz or Buzz for 15.
3) Starting at a number different to 1 (say 0)
4) Off by one error at the end (going to 99, instead of 100)
5) Forgetting to print out the number when neither Fizz nor Buzz is output (first output is Fizz for 3)
6) Always printing out the number. (3 Fizz)
and that's the people who can code at all.
Then you get people who can't code, and can't even produce a for loop.
Oh and extra points for arguing about the solution being correct, particularly after prompting that there is a bug "Is it correctly dealing with 15?" is a pretty huge hint that there's a bug.
It made realize that I've made all of these mistakes on tasks similar in complexity of Fizzbuzz. I write a lot of scripts in different languages some of them quite domain specific.
Why do I make mistakes? Most of time the mistakes stem from the fact that I do not have to get it right the first time and I may have misread the specifications. As Hemingway said: "The first draft of anything is always shit"
So take 1000 applicants who can code but are in a stressful situation and I would not be shocked that there would be 50 mistakes on something so seemingly trivial.
The FizzBuzz test exists to quickly filter out candidates who really cannot code at all, despite what they say in their CV, what jobs they have done in the past, or what courses they have passed.
If you can show that yes, you understand what for loops and if statements are, and can write 10 lines of code roughly correctly, the test is passed and the interview can continue with more interesting questions.
If no, then it saves everyone time to just cancel the interview.
Another interesting data point was that only 2 of the 10 candidates had ever heard of FizzBuzz. That surprised me more than anything else.
Another interesting data point was that only 2 of the 10 candidates had ever heard of FizzBuzz.
Having said that, we tend to pair program for our interviews. Our current interview system is to do a quick telephone interview and then follow up with a 4 pomodoro pair programming session. Usually we ping pong and offer to drive first so that the candidate has a chance to settle in. Generally we TDD a fun kata.
I've been surprised at how effective this is. For the people we've hired recently, the programming session really shows what the person is like to work with (to be fair, we actually pair program on most work so it is easy to judge the effectiveness of the hiring system). Apart from that initial telephone screening, we've dispensed completely with formal interview questions because more than anything it was biasing us away from hiring people with qualities that we cared about (being able to program, being able to pair, being able to TDD).
I've had the experience of cutting interviews short before, when it was obvious that the candidate was well out of their depth. We've stopped doing that. We go through all 4 pomodoros with each candidate. Even if we are sure it's a poor fit, it's important to give something back to the candidate who has taken time out of their day to come and visit. It sometimes turns into a 2 hour coaching session, but that's fine. Hopefully it helps the candidate sometime in the future.
It's kind of expensive, but trying to have an interview system that leaves a positive feeling should eventually lead to a good reputation in the industry. That, in turn, should make recruiting easier/cheaper. If you leave the impression, "those people are awesome -- I really want to work there", then it's worth the added expense, I think.