I'm not sure that this the way a healthy discussion should work. I feel that people with opposing views should be allowed to conduct civil debate without either side being vote-bombed for wrong-think.
It seems really odd that one could lose privileges to participate in this forum for having posted a civil yet unpopular (with some faction) comment. Particularly when the same comment offered as a new post would (could?) not result in the same punishment. Unless of course posts can also be downvoted and I just lack the rep to see that option.
For example, the ability to downvote happens at around 500-ish rep, so users in that point range might potentially see their privileges come and go based on comment down and up votes. While users with more than 4k rep like yourself might never see a change in permissions.
In the end, I accept the rules and workings of this site and gladly participate.
Without focusing on point hoarding (as you say that becomes arbitrary and ultimately meaningless), I think that potentially removing someones privileges with respect to this site should they fall in a critical point range and comment something about OSS that a vocal group on this site disagrees with seems wrong to me.
I think Hacker News participation and the community should be able to have a healthy debate/exchange on tech issues without needing to worry about "points" in any capacity.
You can say what you want (within community standards), but that doesn't mean you can say what you want without reaction from the community.