White-boarding in a high pressure situation is simply a different "skill"... I think it's fair to say that most coders are "internal" thinkers / introverts, that do their best work when they have a chance to quietly think about a problem.
On the whiteboard, you're asking them to do the exact opposite.
Eliminating the "brainfuck" white-board questions doesn't mean you need to soften the interview. You can actually give a more substantial take-home problem to your final round of candidates.