I don't agree. I read HN constantly.
As far as I see, the areas where downvotes tend to occur are where they skew to opinion, or have some tidings to politics. In other areas, HN comments are generally high quality, and have a high signal-to-noise ratio.
I wouldn't also correlate expert opinions being downvoted as just because it would "clash with the mental picture of a number of HN users". That's a false correlation.