Isn't there a maximum of 4 downvotes?

I wouldn't worry so much about it. This is an opinionated group, and having a different opinion is sometimes not taken well, but it is good to challenge the status quo, and good to not be part of groupthink. If you stand by what you said, take your downvotes with pride.

On the other hand, claiming something that isn't true, or being flippant or not contributing to the conversation will also result in downvotes. And those are probably a good guide of how to adjust your communication style to work better within the group.

I have no idea what your original comment was, so I don't know which of those scenarios is the case here.

If the points and resultant behavior shaping really get you down, check out this stupid simple Chrome extension: https://github.com/fapjacks/antihnbs

All visible point totals become 1, and all shades of grey become black.

I agree, however the fear of downvotes (however irrational that fear may or may not be in an online forum) does cause me to stop and think of how many different ways a comment can be perceived. The result is that if a comment contains anything negative, I'll reword it as much as possible so that it can be expressed without causing a feeling of negativity in the reader. Which, by the way, has made me a better communicator overall, and has really helped out in my career.

Edit: I'd also like to see a voting system where people get rewarded for upvoting, and have to make a sacrifice to downvote. For example, if you upvote a comment that others also upvote, you get positive karma, and negative votes can cost you karma.