That doesn't align with my personal experience at all. I do switch between English and Polish keyboard layout on a regular basis (and I mean proper Polish layout, the semi-forgotten PN-87; it's QWERTZ-based, but much more distant from vanilla QWERTY than so-called Polish Programmer's, especially when it comes to special characters). I'm really quick with both and I don't find switching inconvenient in the slightest.
So it's really individual thing and sweeping generalizations like "constant mental switch won't bring you any improvement" seem dubious to me. The truth is that YMMV.