Not necessarily. It's possible to splice together compressed video streams without reencoding them. See https://trac.ffmpeg.org/wiki/Concatenate