It definitely adds latency, not enough to be a bad experience.

We have vdo.ninja today and Twitch's Stream Together. Those both do the 'dual encoding' and it is a good enough experience that users are doing it!