This assumes that your soft skills evaluations are more colored by implicit bias than your technical evaluations. Everything is colored by bias, it's just different kinds of bias. And stuff being inside your area of expertise is no defense against bias. You might subconsciously assume someone doesn't know certain technical things and simply not bother to ask the question.