How do you use this page, if you do?
[0] https://news.ycombinator.com/newcomments
Otherwise, comments out of context do seem pretty useless.
Early when I didn’t have a lot of text I tried the new comments RSS feed and mixed those in with other RSS articles. I had the same problem you had.
I still have the top comments RSS feed going into the system, it is better but still not that great. The highest ranking comments that it shows me have a predicted probability of 0.3 that I’ll like them compared to 0.7 or so about for articles about the NFL and 0.2 for articles about the other kind of football.
What I’d say is that some comments do stand on their own, but many don’t. There is the dependence of the comment on the prior conversation. Some comments make sense without that but you may find the comments to that comment will change the meaning for the comment for me. My take is that the bag-of-words classifier I use for articles is not so good for comments because it does not take context into account.
The default page is heavily biased towards the common denominator of technology understanding and intellectual curiosity, and it is also manipulated by people with a strong left leaning bias.
The "new" tab is where I find a lot of the more domain specific tech articles and also some interesting topics worth paying attention to before they get censored.
But as a general rule it isn't a conspiracy, it's just people who agree with each other and disagree with you using the site without coordination. And if you have evidence they've formed a voting ring or a flagging ring, then email dang about it and he'll fix it.
If you can't find any evidence - it's probably just people who disagree with you. A good test is to ask yourself, "if this was all above board and there were just a group of users who disagreed with me, because they had independently formed a different opinion than I had, would it look any different from what I'm seeing?"
But that's really only interesting briefly. The fun thing to do is find somebody who makes intelligent and constructive comments and then trawl through their comment history for other good comments. YMMV
I tend to think of HN as 3 overlapping communities: startups, programming and other random stuff. Lately I care only about the latter 2.
I tend to go Ask HN first, then my threads, then main page, then Show. But a lot of the good stuff is actually on the 4th page of news - things that are not controversial enough to get back and forth conversations, but interesting enough that someone comments on them after they're no longer featured.
This yields links and discussions I would otherwise miss, so I still do it sometimes. YMMV.
Edit: mixed "new" and "comments". Fixed.
Always instantly regret doing so