For sure; that's how this typically happens. It's one reason why the guidelines ask you not to post such comments about voting. Vote totals fluctuate. When users see an unfairly downvoted comment, they often give corrective upvotes. Then the comment goes back into positive territory, but a post like
https://news.ycombinator.com/item?id=20057244 lingers on, orphaned and inaccurate.