You're assuming that HN isn't keeping track of how often you downvote (or any patterns). Without knowing the set up, it's difficult to infer much. Of course, revealing that info might lead to gaming the system.