If you spend enough time in those threads to notice patterns, and even look through people's posting history, you're essentially acting like a bot crawling the site.
If you then downvote those posters you disagree with, it creates a very suspicious voting pattern on their end.
Could you distinguish your own behavior from that of a bot (or a bot-assisted human) if you didn't know it was just you in front of your screen?