And that’s why there is SponsorBlock
I admit I've marked segments on videos, e.g. skipping cringy jokes when I find the presenter annoying.
If so, then it is a plain case of internet vandalism. I would just ignore such segments.
But if the segments were marked correctly and there were too many of them in the video, that is the problem caused by the video creator, not by the community which publishes the segments.
I use SponsorBlock and haven't had any issues. I enjoyed when it skipped 99% of an MKBHD video, it was kind of funny.