It's just a bit frustrating hitting the wrong key? It's not more complicated than that.
Mistakes derail me much more when I'm sharing my screen or pairing. Once I make one, my confidence is somewhat shaken and I spend an inordinate amount of time on accuracy rather than the task at hand. Stupid, I know.
Yes, it's detrimental because it's distracting. It interrupts my concentration when I make typing mistakes, or can't get the code written down fast enough. I want to write it down fast, because I can only hold so much in my head, and the faster I can write down an idea, the faster I can move on to the next one. Once everything is down, it's much easier for me to look at the whole, and edit it (refactor, etc.).
Perhaps it's not detrimental, but I can also program on my smartphone without it causing major problems for the end result. It's just super annoying.