That's one of the reasons that they are best done by experienced HR personnel.

Good HR people are worth their weight in platinum. I used to work with one whose thumbs down became an automatic "no" from the team because we discovered that she was so good at reading people that everyone she didn't like inevitably threw off massive red flags in the rest of the interviews.