I've noticed this with Skype. It's pretty funny: you'll not hear much for a few seconds, and then hear the other party speak really quickly, with all the pauses between words dropped out.