A steep rise of Hacker News in Google rankings
jonathanpagel.com
jonathanpagel.com
Unless they can make a reasonably effective LLM-generated content detection (sounds tough!) they're faced with a rather inconvenient problem of mining the internet for known human-generated content (while we still agree what 'to know' even means).
Interesting parallels to https://en.wikipedia.org/wiki/Low-background_steel.
Yes. For now. Just flag the use of the words "delve" and "deep dive," flag bullet-pointed paragraphs and lists, and flag anything that closes with a too-neat one-paragraph conclusion — and you've just caught 85% of AIspam.
But this is already changing, and in about 6-10 months I expect it'll be impossible to detect at more than 70% confidence whether an article is written by an AI or a human. (You could say: The human writes with soul, with style, whereas the AI is totally bland, irritatingly didactic, and generic. But you can quite easily coax an AI into imitating any literary style, and some of the best AI-generated content is warped in this way.)
Uhoh, is my copy pasta in danger of being snuffed out before anyone is served it?
Look, I get how some people would consider it spam. However, the problem still remains and the solution has not changed!
I have a life to live and can't spend every waking moment crafting a unique comment every time I need to reply to someone.
They had an algorithm that perfectly recommended content to you. Then they scrapped it and went back to star ratings. Why?
Netflix makes money from monthly subscriptions. They lose money when actually streaming content to users. So the ideal user from their standpoint is one who keeps paying but watches almost nothing.
Thus, the company is incentivized to put out just enough good stuff so you won't cancel, but they don't want you binging.
If Google sends users to pages and advertisers pay them, the incentive to address the content problem isn't there. In fact, LLM content may be another cash cow waiting in the wings.
The prize was a recruitment tool from the start, though. I don't think they ever intended to use any result.
For example, the RefinedWeb paper lists HN as one of only 12 websites that were excluded. From what I understand, it was excluded because it went into the final dataset unvetted. RefinedWeb was used for the Falcon model.
Same is also true if you're implementing half of your moderation at client level.
4chan: the roaches of the Internet :)
I haven't really bothered going back, I'd like to say I've grown up since being like 14 and getting pretty accustomed to seeing some pretty terrible stuff on there.
God I love hacker news… their insanely outdated moderation tools are a shame, but I can’t lie, holding a big stick makes for a peaceful forum.
(hi to those from GitLab watching #hn-mentions on Slack)
That's really interesting that they'd disclose their strategy so publicly. On the one hand, I can't see it doing much harm. On the other hand, wouldn't it be somewhat self-defeating or even embarrassing to confess that you're doing that? One could probably reasonably assume that many companies have a social media strategy like that, but to have people know that's the strategy would probably drain away at least some of the goodwill provided by the posts.
bad way - posting stuff to make your company profile go up all the time, if you think you have something that might be of interest to HN sure but not everything you do should get posted. when negative stuff comes up swoop in to defend company and to drag down those saying negative things about company, especially without disclosure but even with disclosure.
Good - something technical about company comes up, developers who worked on technical thing come in and clarify technical aspects for people. Somebody has problem with your product and you come in and ask for clarification and help solve problem.
Someone should build an AI agent that keeps a list.
If you don’t want something to be in Google on a policy level, you now have to add the noindex meta tag on every page.
A site is spam-proof if it makes it hard enough to get an invitation. The relatively few spammers who slip through can be warned and/or banned.
Unfortunately, we may not be invited.
Being open is hard, very hard, but it's the only long term strategy in my eyes. The rest is an illusion.
As evidence against: Lobsters, which uses an invite tree, has virtually no spam to speak of.
That being said, if there are lots of dormant accounts on lobsters, they are likely spam bots just waiting to be activated once the rules are deeply understood and there is something to gain, or be waiting until there are more than the mods could actively fight against because they lack the tools to do so (see: false sense of security).
It's pretty clear that invite trees provide actual value and the opposite of a "false sense of security".
I'm simply telling you how spammers work on OTHER platforms, quite successfully. Whether lobsters is immune to that or not, I don't know, nor do I care to know. If you think they are immune, then great (see: false sense of security).
Users who actually believe in the site and understand how the mechanism works and appreciate the high snr that they've enjoyed, will be understanding for the most part even if they end up on the wrong side of one of these bans, and have to go through an appeal system to rejoin the site on some other trust tree which is much more closely monitored by its members than the one they were on before. Ensuring that future bans will be highly unlikely.
Depends on whether you can trust them. If someone at the top of the invite tree gets hacked or starts shitposting, it doesn't really imply anything about the people they invited, they're all probably innocent users. If they turn out to be a spammer though? Then you can't trust anyone they invited. Any false positive should be handled on a case-by-case basis or reinvited by someone else who is trusted and is willing to vouch for them.
Thanks for answering my questions though. It was insightful.
The other commenters remark that you ban the entire space below the spammer, is a subjective judgment call they would make if they're the one paying the operate the servers and not wishing to use their time and energy to subsidize someone's desire to abuse the internet. Acting like this is some sort of petty fascist power move is not arguing in good faith, it is actually an attempt to escape an argument that you are failing to understand, by changing the subject from how to deal with a certain kind of common social network pathology, to a completely different name calling argument about someone's imaginary nepotistic motivations. It's probably best that you do excuse yourself from such discussions, even if you feel you need to disguise your exit with a juvenile parting shot.
Anyway you already advocated for, as you say, a "nepotic" system, by presuming to kick out people - spammers- whose behavior you feel disrupts the environment you're trying to curate for the benefit of people who are there to support it. There's absolutely no pretending you're above the enforcement of value system boundaries, it's only a question of how focused you want them to be, what stable values you want to optimize for in order to create a consistently attractive experience for your users.
I use this site far more than reddit, in large part due to the fact that it does encourage its users to identify and punish behavior (through flagging, disliking) that is out of step with the objectives of the site, as outlined in the usage guidelines which everyone implicitly agrees to (try to) follow when posting here. And yet in the public scope reddit is the clear 'winner' if raw traffic metrics are the score.
After being on the internet for 20 something years and Usenet for 10 before that, I've had quite enough experience with free for all dumpster fire communities to know that they are not the bastions of high-minded openness that people naively or foolishly believe them to be.
How are people lying on the internet and me having a discussion about it somehow me acting in bad faith? I've made it pretty clear in this thread that I don't care for lobsters, but I haven't made any arguments, at all, in bad faith. If anything, I'm trying to help the commenters by informing them of how spam networks work. So far, I've only been told that I'm wrong. So, whatever.
And yes, treating the nodes closer to the root with "favoritism" by not banning all their children and then banning "further away" nodes by banning their children (and parent?!) is almost a textbook definition of nepotism if nepotism extended to trees.
Being suspicious of dormant accounts, is not nepotism.
You haven't informed much to be honest. Whenever you're pressed for real bits of information, you just say you can't reveal the facts because they're secret and revealing them would help the spammers. So you're not doing much to reduce the uncertainty surrounding this topic which is critical to quality information. You're just generally claiming that the methods we're describing won't work while providing nothing but appeals to authority.
The information you've provided is the fact spammers try to obtain invitations and create aged accounts in order to look legitimate. This is already known and is the reason why you turn accounts and invitations into scarce resources to begin with. Spammers should not be able to just randomly create accounts, they should need to be invited. Whoever invites someone is vouching for them and responsible for their behavior. People should be inviting good people who they trust, not random spammers on the internet. You enforce this by also punishing the guys who invited bad people into the community.
I'm not claiming you're wrong either. I'm defending the rationale for the methods I've seen actual administrators employ to manage real communities. Those guys weren't up against spammers either, they faced far more serious adversaries.
So sure, this thread isn’t very informative, but that’s not entirely my fault. I’ve said as much as I’m free to say, the rest relies on other creative people who can read between the lines and ask the right questions.
At the end of the day, I’m sure some of the people that have joined the lobsters fraternity, er, site, have at some point worked at trust+safety in big corps and can actually help if asked. So I suspect they would actually be fine. I was just hoping to have an interesting discussion about it here and help people understand that a magic tree won’t help you in the long term.
If the problematic account is too high up in the tree, then they probably have way too many descendants which increases the false positive rate which means you can't conclude that they're bad based on the fact they're invitees of a bad account. Essentially, the spammers diluted themselves inside a huge number of maybe good users so that you can't blanket ban them without collateral damage. In that case you're better off banning the bad account individually and looking for new bad roots further down the tree.
Let's look at the definition:
> the practice among those with power or influence of favouring relatives or friends, especially by giving them jobs
Basically, if you are friends with the "right people" (aka, people closer to the root), no matter what they do, you are fine. You aren't getting banned for one of their actions. That is people in power (closer to the root) favoring friends (the people they invited), where "power" is the ability to behave with near impunity without severe repercussions.
Make accounts and invitations a limited resource. People will think twice before inviting someone who can get them banned. This reduces the problem to one of trust which is the foundation of real security.
Yeah, but then it's also invisible to search engine users, in which case no one will want to spam the site for SEO anyway.
I have to admit I agree. I really like the benevolent dictatorship which does not allow that kind of BS, even from me. (The time dang gently reprimanded me for some kind of sarcastic attack, upon reflection, I decided that I liked this high standard, and resolved to do better.) I’d hate for it to change too.
I really like what lobsters did with invitation tree https://lobste.rs/about#invitations
Best $10 I spend every month.
I understand that the average HN commenter puts more effort into their comments, and their veracity, than the average internet user elsewhere, but still.
(For me, the top "result" for Monaco in italian is the Google Translator widget with Collins Dictionary, Wikipedia and Quora the top 3 links. HN is 4 or 5 links below.)
Oh, and yes, let's hope spammers don't overwhelm dang!
When Google arrived, it had the solution people were desperately waiting for. It was pretty much everything we wanted. And it even "wasn't evil!"
Now it feels like people at Google are just making sure they'll qualify for their annual bonus with total disregard to what happens to the company.
I forecast that in the next 1-3 years, we'll see another company steal the search market, just like Google it back in the day. But Google, don't worry, they'll "just be a search company".
Really? Do you think this company exists today?
Remember that what became Google was Larry's and Sergey's PhD project that no one wanted to buy.
Kagi is a very strong and viable candidate IMHO. The search is actually useful and I’ve noticed on a few occasions that my search was “too focused” and yielded “no results found” and I had to rethink my query to better find what I was looking for. Google on the other hand would have spared no opportunity to spam my results with ads even if it couldn’t find what I was looking for. Anything to put more ads in my face.
Why Google have not bought reddit, I don't know (beyond moderation issues, but AMZN made it work with twitch).
Maybe I'm just really paranoid these days, but I would bet looking at searches with Reddit in them and creating threads or commenting on old ones and paying for up votes is probably lucrative.
where does this notion come from? google is for them to find out what you think you want so they know what ads to serve you. if it was for finding content, they would show you the results that were actually related to your query.
On the other hand, if google owned it, getting banned from a subreddit would possibly mean getting locked out of all of your google accounts.
They also don't quietly remove comments in a way that is invisible to a user for triggering some keyword in AutoModerator or a spam filter. And there are usually no minimum karma or account age requirements for posters.
Any large site does this. Even HN.
Moderation of that type usually seems to be secret, but the problematic moderators that I have noticed seem to be trying to protect some political belief or pet disinformation from discussion. It poisons the entire site for me.
For example, I wanted to post a funny Risitas youtube vid I made (you know the Spanish comedian with that laugh...), and couldn't find a single usable "funny" subreddit. Some banned youtube content completely. Some banned "video memes", some banned X and some banned Y... all of them had slightly different guidelines and you immediately got an insta-splurge from a bot-mod if you tried posting. Some required you to prefix every post subject with some code word. I had to give up eventually and post it on some super-small subreddit instead that accepted anything.
My sense is that the bigger the userbase, the more it attracts junk, spam, abuse, etc. So, the rules get tightened to combat it. Also, my impression is that the moderation tools are not great, so crude/heavy-handed methods are sometimes all that is available.
I think you found the corollary already: smaller subreddits have less rules and/or less strict enforcement.
I'm not sure a better solution, given the situation. Though I agree it can be discouraging for well-meaning occasional contributors.
"... It is a well-known fact that those people who most want to rule people are, ipso facto, those least suited to do it. To summarize the summary: anyone who is capable of getting themselves made President should on no account be allowed to do the job."
— Douglas Adams
He was Spanish, as in Spaniard, from Spain, Europe.
Your comment has been auto removed for not linking your reddit account with a email.
Sorry I am good at leaving sources to backup what I say I guess?
I mean does anyone actually like Reddit anymore? Front page is almost entirely bots reposing and reusing the same comments that have been used for years.
I do like the good parts, deeply hidden, that can often be surfaced with a Google search. Case in point, I had a bug with some installed software yesterday, to which the only viable solution I found was in a Reddit post from a few months back.
But the experience of actively browsing the "leading edge" of the site? Absolutely not. I purposely-deprecated my credentials a year or so back and haven't regretted it.
Yes, but if there was a better alternative I would switch in an instant … but those network effects.
My guess is that too many people at Google are on Reddit and didn’t want to see it go to the Google graveyard the day after acquisition.
I was a moderator of some very large subreddits, and due to reddits pigeonholing me into an app vs new mobile layout, I'm leaving those moderator positions (note: I am not complaining about the api issues). I don't want to participate in a community that is catering to the lowest common denominator such that the term "redditquitte" is a joke.
I've thought long and hard about it, and I think companies are intentionally creating Eternal Septembers in their products, because it's just easier to just put big pictures on the homepage to get clicks, when that type of UX only invites the type of people who see the site as something only to consume and not to contribute to.
I've been invited to multiple "moderator feedback" focus groups, that were worse than awful. After they defaulted an "annoying look here" icon in the right corner to try and get us to work more, I said "fuck this, I'm out."
My point here isn't just to bitch and moan, it's to point out that site:reddit.com only works because the community is one that actively values contribution over consumption... that's going away, and the usefulness of site:reddit.com will go away as that culture changes.
I guess the mobile app was what really broke the camels back but the quality of the posts had been on a downward trend in a severe way since at least Obama's second term, when I think both political parties recognized it as important and began to manipulate it. This is made easier by the partitioning of the site into subreddits. I hopefully don't have to explain here why that makes automated sockpuppeting much more effective and easier to accomplish. It's a fundamental design flaw (if we were to assume the design of Reddit was intended at all to provide a space for authentic personal takes on real issues and by real humans).
There is a danger of the same thing happening to HackerNews but I hope the lack of financial incentives to allow that sort of thing does some work to mitigate it, along with the lack of partitioning of the community.
You can easily ignore the vitriol and fake news on Reddit, you can not get around a lot of commercial detritus anywhere else anymore
Huh? I don't follow here: having multiple subreddits is exactly what makes the site usable for so many utterly different niche topics. There's probably a subreddit for repairing 1967 Camaros; do you really want to see posts like that every day in your news feed? I don't. Reddit isn't meant to just focus on tech topics like this site; it's meant to be a site with discussion forums for every topic imaginable, and there's no practical way to do that without subreddits.
Any attempt at pinpointing the enshittification is bound to be extremely subjective. What is clear is that it has been a continuous decline for a long time.
I don't know that the Twitch acquisition was smart. What was the strategic reason to acquire?
Amazon announced cutting 35% of Twitch staff (500) in January to stem losses after two rounds of layoffs last year.
Looking at those video lengths, I'm inclined to believe you.
Any video that is 9 to 11 minutes long, I automatically skip because that's the sweetspot length for maximizing ad revenue. This person's are a bit longer, but looking at the video subjects, I can't see why they need to be that long.
I think that may be outdated? 10 minutes was the sweet spot for ad revenue a few years ago, but I think around 8 minutes is the sweet spot now.
In the end his omission of other factors have led to a conclusion that isn't entirely accurate. (That Zerowater is the best because it filters TDS to 0).
What about bacteria? Other filters that faired poorly in the TDS test focus more on that and then there are a few major brands that weren't even featured.
What about clarify of water? He did touch upon this very briefly but did not go further into it. You would think that a TDS of 0 indicates the water is perfectly clean. This is wrong as TDS cannot measure everything that may be in the water.
There is more to water purification but these are just two examples.
All in all, I have been impressed by Todd's reviews but after watching this review where I actually had done some research into the topic before watching, I came away doubting all his other videos.
What did I miss just because I am not a subject expert in the topic?
I guess at the very least his videos likely eliminate the very worst products but I bet you people are buying whatever products he recommends without thinking about them and may be getting burned or not getting the best product for them.
These are different suggestions based on assumptions of your situation. Assuming you have access to fairly decent tap water (Europe and US) I'd follow Todd's recommendation for Zerowater as thats what I daily drive. I used to rely on Brita but it does not do much...just slightly improve taste and remove the worst heavy metals.
Running tests on my water, there are still traces of some impurities like bacteria and some metals after Zerowater filtering.
I am also about to start testing the Aquatru system that he also recommends.
For Zerowater the filters are quite expensive (15$ per filter) and I have been peeved at how they get used up quickly (I have around 200PPM in my tap water and they last 75-90 days) and still dont filter everything out fully...but subjectively I can't get over how I like the taste and since im not on the west coast im in the more lucky group where Zerowater is good enough. Maybe Todd was also in this situation?
My family does not like it but I love it. It tastes like "flat" water and becomes filled with bubbles if let to sit out for a short time but its worth it because it "feels" so clean.
Sorry if this observation is unscientific but once you have physically removed impurities, you still will have some semblance of subjectivity. I try to target distilled water taste as a point of reference and Zerowater comes close.
I have worked on a system where I first filter out using a Brita filter and then run it through the Zerowater to help improve the results but the problem with this is that my water is not bad enough for the brita's simple activated charcoal to really help reduce so in my case it actually has ended up giving mixed results.
If you lived in the west coast where the water is typically around 400ppm then you'd extend the life of the Zerowater filter quite a bit by doing this trick but for me, well I have to try something else.
But at that point maybe the Aquatru is better which is why I am trying it. For me, the Aquatru is just for curious comparison as this whole journey is reaching nutjob levels for me at this point and im not elon musk levels of rich(these water tests are not cheap)...the Zerowater is good enough for my usage because I live in a suburb away from any industrial places/poorly managed municipality(no major pathogens, just correcting the taste/eliminating any traces of metals and dirt).
In reality, Brita is probably fine but I want that taste of flat water now that I have gotten a craving for it. Every time I drink something else like Brita or bottled water it just tastes weird.
In reality others have told me if one is to spend the dough on Aquatru, you might as well get a under sink reverse osmosis system installed. Takes up less room and is cheaper. I got a good deal on a open box unit so I decided to go that route. Sorry I dont have results yet.
If you are concerned about pathogens that could make you sick, then it becomes tricky. I have traveled to Pakistan and lived in places there where the tap water makes you sick. I have relied on Grayl and based on my testing water and sending it out it seems to filter pathogens but unfortunately it is a massive pain to use. Do not rely on Zerowater/Aquatru for this as it will not help you. I am still in the search for an excellent under sink solution to eliminating pathogens + giving me the taste that I get with Zerowater. My ideal combination would be to have some sort of automatic Grayl + filtered afterwards with Zerowater. Beautiful tasting water but quite expensive. Might as well rely on water bottles at that point. Hope this helps a bit.
And dont get me started on the changes they made if you search a product or something buyable. its takes like 4 clicks to go to the shop page and none of the open-in-tab methods work
before/after:<year>
Combine this with Google now placing only sponsored content on damn near the full first SERP for some terms and it has become less useful by an order of magnitude (I meticulously calculated this figure).
And, this degradation at the same time ChatGPT has come on the scene. I know I personally bypass Google altogether now more and more frequently in favor of ChatGPT. Wonder how many people do the same and whether there is a whiff of desperation at the Big G.
Would you be willing to consider the possibility of delisting the site with robots.txt?
(BTW, if this is gonna be done, we should also block all AI bots and search engines at the IP address level.)
That's not possible or necessary.
Not possible, because it simply isn't possible in the general case to differentiate real papers from scrapers, without using device attestation. For an extreme example of this, see the Recap the Law project[1], which gives real human users an extension which scrapes as they browse.
Not necessary, because scrapers for AI training data an entirely separate problem completely unrelated to marketing, and because robots.txt will serve to stop the majority of search engine indexing, which is all that we need. Actual blocking of engines isn't necessary, because all the big ones are well-behaved, and after they stop indexing HN, marketers won't care about HN for the purposes of SEO-related influence campaigns any more.
Email it is!
Browsing HN is a relief in times of Surveillance Capitalism
Edit: HN uses google tagmanager and analytics; amplitude and branch.io
We do? No we don't? I don't even know what those are.
If you see requests outside of that there's something fishy going on in your browser, I believe.
The linked blog post actually describes that Google is drowning in ad-driven crap, and has to rely on manually chosen external structures to keep quality tolerable, not that “HN has became cool”.
Profiling other users, in addition to being extremely bad form, is a pretty good sign that you don't actually have a valid argument to make - which, reading your comment, you don't.
My language was "Would you be willing to consider the possibility" - a polite request. There's no way that a logical person could even read that as either a command or like I was acting as "the voice of the majority" or "a plurality of users" - you pulled that out of thin air
Dang is not a tool or an idiot - if he thinks something is a bad idea, he won't do it. If he thinks that it might be worth doing, he'll think about it. If he needs to poll users, he will.
My entire life it's been my experience that the less people involved the better things are in general. There's probably an optimal number of people for every community. I don't really know what that number is but I seriously doubt it's in anyone's best interests to exceed it.
> I would like to make the argument
Which makes this perspective very confusing:
> it seem like you were the voice of the community
HN is an experiment. As a rule, a community site that becomes popular will decline in quality. Our hypothesis is that this is not inevitable—that by making a conscious effort to resist decline, we can keep it from happening. - https://news.ycombinator.com/newswelcome.html
pg wrote that over 15 years ago. I've been saying for (a mere) 10 years that we're trying to stave off the arrow of internet entropy: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que... – not doable forever, no doubt, but doable for a while if we expend enough energy.
Based on experience so far, the way to do this is through a combination of a dedicated community, software that does what software can, and moderation to bump the system out of its failure modes.
Trying to hide from the outside world is mostly not the way. pg used to do tricks like Erlang Day on occasion, but having that as the main strategy would be like trying to avoid infection by never going outside. Far better is to have a robust immune system, if possible. Trying to avoid (resistible) pathogens weakens the immune system, and hiding HN from new users is a path to decrepitude. The latter is probably a greater threat IMO, and too easy for established users to discount.
Spam, and its cousins like content marketing, could kill HN if it became orders of magnitude greater—but from my perspective, it isn't the hardest problem on HN. That's because of the dedicated community, which flags these things, reports them when they escape in the wild, and is vigilant about quality. Without such a community, HN would have died long ago.
By far the harder problem, from my perspective, is low-quality comments, and I don't mean by bad actors—the community is pretty good about flagging and reporting those; I mean lame and/or mean comments by otherwise good users who don't intend to and don't realize they're doing that. There's an unholy dynamic between those and the upvote system, so worse comments often get upvoted more than better comments do—often enough to choke the threads with weeds. That's the high-order bit and what I spend more time worrying about—not Cassandra E Oakley and her trading system*, nor the latest startup voting ring and whatnot. If those ever become the high bit, we might be doomed, but we should see it happening long enough in advance to readjust.
p.s. (@dang doesn't work - I only saw this by accident. Well, not by accident because it ended up at the top of the thread)
* https://news.ycombinator.com/item?id=39425371 - unkilled for the occasion
blame those dang programmers!
I agree it probably doesn't make sense to try to block growth sources though. Just considering the perspective "it's not effective to try to play whack a mole anyways" already seems enough evidence of that alone.
All this reminds me though I've been using the site too much lately and my interaction quality has gone down. Time for another short break :). Anyways
So many places on the internet used to be wonderful places of discussion. I remember as a teenager I would come to the internet and be in awe at this system that mankind created to allow for all humans to come together and discuss.
But I’ve noticed since 2015 every single discussion place I used to frequent online has become horrible.
I discovered hacker news only recently and this website seems like the last remaining jewel of the internet that still exists. I think the reason is because of you and the philosophy you use to moderate the website.
Just wanted to express my gratitude to you. Not sure how often people say thank you to you for the work you do but they should say it more often!
At first I thought it was a site entirely devoted to computer stuff but it was only later that I realized what this site was really about.
Can you elaborate on this a bit? I don't see why "lame and/or mean comments by otherwise good users who don't intend to and don't realize they're doing that" should have an "unholy dynamic" with the upvote system.
(FWIW, what I have observed is that once a comment becomes established as the top comment in a thread -- and it doesn't take much for that to happen -- it is nearly impossible to dislodge it. That means that getting into a thread early is crucial for getting noticed. I've pretty much stopped commenting on threads that are older than an hour or two because I can be 99.9% certain that whatever I write will never been noticed no matter how good it might be. And FWIW2, the comment I'm responding to is 50 minutes old as I write this.)
Sometimes moderators actively bump down top comments.
It’s probably related to what dang wrote above about the “unholy dynamic” of low-content comments that trigger easy upvotes. I remember myself having written some (frankly) lightweight low-effort snark, see it end up at the top of the comments for a few hours, and then it went mysteriously halfway down the list even though the upvotes didn’t stop.
To be clear, I think it’s a good thing the moderators do this kind of weighting, and the invisibility of comment upvote counts to non-authors is an important feature because it enables this.
People optimizing for this kind of low effort but highly upvoted comment is called “karma whoring” on some places.
The reflective circuitry, which takes in new information, turns it over, and generates an unpredictable response, is much slower and harder to run. I suppose it's a bit like the difference between a sugar hit and eating nutritious food with fiber. The latter makes you feel better in the long run, but when it comes to mass dynamics, the sugar hit wins out every time.
> once a comment becomes established as the top comment in a thread -- and it doesn't take much for that to happen -- it is nearly impossible to dislodge it
Moderators downweight top subthreads that are generic or otherwise lame, and repeat this until the top subthread is no longer lame—if possible. The trouble is that this is an intensive manual process. Most likely the software needs to be adjusted as well.
[1] https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...
[2] This is probably the basis for the generic subthreads which are the bane of this forum: not bad enough to flag, but predictable enough to suffocate.
context: i read the "past" page every day, so often am several hours or over a day behind discourse. interstingly enough this post is still relatively new and was popular enough before midnight utc to show up in today's past page.
i wonder how much positive feedback posts receive if they miss the initial window.
I know HN does something similar, but it is not quite as good as reddit. From observation, specifically the 'penalty', or added performance requirement, of latching on to a top post is too weak. The result is that all HN comment threads consist of only a few top level posts, with subthreads growing off them, because you can easily 'jump the queue' just by commenting to a top comment. This is also what contributes to the idea that it is pointless to make new root level comments after an hour - because almost all the action is in subcomments to top comments.
Edited to add: Reddit soft-hides (collapses) subthreads that are deemed lower importance, which is probably key to make the ranking system work. Anyone interested in a subthread may expand the hidden/collapsed sections, and they may even be upvoted back to uncollapsed state. But by default they don't muscle into the main conversation. HN already has the collapse feature, which could be reused for this. It's just a client-side collapse, also the reddit one (though in huge threads, deeper threads will be loaded on demand).
How?
> the 'penalty', or added performance requirement, of latching on to a top post is too weak
I'm confused by what you mean by 'quality' and 'performance', unless you just mean upvotes.
I think reddit just counts a ratio of upvotes to views (ofc downvotes too). It is possible that users collapsing a comment/subthread also has some weight. Would make sense.
> And, importantly, the quality requirement for the post increases the higher in the total hierarchy it is: root-level post, top-level reply to a top comment, and so on
What I mean by this, is that the more prominent a place a post holds, the better it must 'perform' (per the above definition). Prominence being mostly just how high on the page it is.
considering this along with the comment below in the thread:
>most activity on a thread seems to happen in the first 24 hours of posting it. the discussion tapers off beyond that.
Supports more software experiments to encourage "slower" or longer discourse, and thus reflection.
This is my longtime wish as a user: reading via RSS most conversations are long dead when I get to them. I subscribe to replies on my comments, but since few other people do, it's not that useful. Simply promoting such features might help, as I bet few users are aware of them.
1) my comment actually did capture your attention 2) I did notice it (courtesy of HN Replies)
The current state isn't working too poorly. and sure beats the pants off any other option i know of ;)
So please keep up the awesome work.
And the other problem is that even when it works perfectly, this process as described can only produce non-lameness at best.
I have a suggestion based on something I did at Google 20+ years ago: compute Page Rank on commenters. I did this 24 years ago in the Google Translation Console, which was a (now long-since-retired) public interface for volunteers to translation Google's site content into other languages. Translators would not just input their own translations but also rate the translations submitted by others. Translators whose translations were upvoted more often were deemed more reliable raters, just as links from highly ranked web pages are weighted more highly when computing Page Rank. This successfully prevented any translation spam from ever making it through to the public site (as far as we could tell). It never reached the scale of HN, but it's a lot easier to implement such a thing nowadays too, so I think it might be worth a try.
How do you structure this to avoid that problem? How would the system work for a new account just created to get established?
Just out of curiosity, who would you cite as an example of a lame commenter with tends of thousands of karma points?
Also, stuff like Erlang day isn’t a bad idea: it is also a kind of nudge to remind the community about what kinds of discussions and comments to prioritize.
> This ratio has steadily gotten worse over the past years
I'm not sure. Everything has always been getting worse, or feels like it has. You'd have to discount for this bias in order to tell whether things have really gotten worse, and no one really knows how to do that, nor really wants to. It's more satisfying to feel the decline of civilization setting in as one ages. Yes, that's a grandiose generalization and therefore lame.
Or, to flip it into a positive, you're more experienced and informed than you used to be, so more comments appear misinformed or inexperienced.
This is a problem I've seen in specialist forums elsewhere; the conversation runs out, because nobody's poking the community with wrong stuff, and correcting a wrong statement is intensely more motivating than just making a plain factual statement for its own benefit (also: when you do that, the rest of the community will go "uh, yeah, and?").
Would be great to see some actual data analysis.
I wonder if that would give enough pause for users to consider if a comment is worth their upvote.
However, while I completely agree that spam isn't anywhere close to being a threat to Hacker News, I'm not so sure about subtle, well-executed content marketing/influence campaigns. By design, these are much less overt than mere spam links to a porn website, and there are plenty of places for them to pop up on HN - for instance, in any number of recent threads lamenting the sad state of modern appliances/tools (like vacuum cleaners, e.g. [1] - and it would be extremely difficult to even approximate the scale of the problem.
I'm not advocating hiding from the outside world in the same sense that one might try to intentionally isolate their community to prevent others from joining - just in the sense of avoiding a specific discovery mechanism (Google) that results in extremely strong incentives for content marketing companies to infiltrate and influence. More specifically, I posit that the amount of organic discovery and community growth that we would get would be significantly outweighed by the attention from content marketers - most people that I know discovered HN through blog posts and individual recommendations, and not through search engine results.
Ultimately, of course, it's up to you - but please think about this issue (and particularly the tradeoff of the upsides vs downsides of being highly-ranked in Google results), and look for indicators of content marketing. Hacker News has, by far, the highest quality-size product of any community that I've seen on the internet ever, and I would dearly love for it to stay that way for many decades to come - which I don't think will happen if "Hacker News product recommendations" becomes a currency with supply comparable to GitHub stars[2].
Similarly to how reply is hidden for the first few minutes for hot threads.
I mean just look at the domain name
HN's pedestrian design makes for a much better experience.
- no flair for usernames - means people concentrate on the message not the messenger
- no visible karma for anyone but the top few - means people don't spend (as much) time karma whoring
- limited formatting, no images or video - increases the value and import of the written word
- no sharing, or @user referencing - means posts live and die more by their merit rather than brigading or other shenanigans
It took one click to see that user abraae has 6671, as of this writing. Or did I miss the point in dramatic fashion?
I won't link it; per the hacker ethos, I encourage you to find it yourself.
E.g. A reply/comment is at the top of the chain, but you don't know if there is a difference of 1 karma or 100 karma between it and the next comment.
At first I thought I wouldn't like it, because I tend to post on subjects I'm knowledgeable and want to do my best there. But now I'm so glad to not get the anxiety and rush of the back and forth in heated discussions.
Peace and quiet.
In HN the username is simply plain text. You can't even see it's a link without hovering.
It’s hate of the new Reddit, plain and simple. Well deserved might I add, it embodied the new priorities of the site: number go up.
I quit Reddit cold turkey when they took Apollo away.
most of the readers here don't mind reading text and face a wall of text daily, so it works out here.
Even for us, we do mind reading HN as a single wall of text. I've got my userContent.css file set up for HN so that:
1. There are maximum sizes to the paragraphs (I can't read a wall of text stretching across 1080 pixels!)
2. Vertical spacing between comments and lines are larger.
3. Each comment in a thread has a larger indent using padding so that, visually, it's easier to track the parent of any given comment.
The case of the filenames may or may not be important, depending on whether on Windows or not.
https://gist.github.com/lelanthran/873983febef21450b0afcb99d...
It won’t improve since when Google makes a change, SEO marketers adapt. The websites that actually provide value and don’t really care about SEO suffer as well as the users looking for that exact information.
I just don't see how anything else could be sustainable.
Curation was what search engines replaced, as the scale of information available outgrew human capacity to keep up. We are almost certainly going to have that problem again soon, for a while at least.
Really, what I hope is that the already burgeoning problem of AI-generated garbage gets solved, and that people rediscover the virtues of social interaction that's based in reality, rather than in the optimization of strongly emotive idiocy that adtech-driven social media demands.
There's probably not more than 50k meaningfully unique sites with some notable amount of actual desirable information, after excluding all the SEO'ed sites, blogs repeating each other, etc... at least for the English web.
Manual curation is entirely possible since probably there aren't even 50 such sites being created per day on average. This is including every single forum still open to public viewing. There really aren't that many left (<10k).
Especially worth mentioning in this connection is https://news.ycombinator.com/item?id=39424688, as of this writing #1 on HN. I mention it here because what it says about moderation, and about centralized platforms being both the highest-value and most poorly managed targets, applies here also.
Why would you expect otherwise, that intelligent people will suddenly lose their ability to perceive what's higher quality content?
The issue is that those are now impossible to find.
A huge proportion have simply stopped updating, gone offline or moved to a paywall on substack/medium/etc...
The 50k number is all inclusive and probably even still an overestimate.
And I never mentioned whether something 'qualify as information worth retaining the ability to retrieve'... are you confused about what the comment chain is about?
I'm okay with that. Honestly sounds a lot better than the current state of affairs. The only problem now is getting invited.
This is not how the good years of the Internet grew. I can't think of a single good or popular thing that started out as "invite only" other than I guess Facebook and Gmail. Both of which were actually more marketing gimmicks.
But I don't think it's bad. Hackers/smart people "locked in the basement' or talking only with their friends in their bubble is not ideal. There are a lot of people out there, with their own opinions, ideas and understanding (or lack of) of the world. Internet just converges towards the average human being. If we want a better internet, hope some smart people will put some effort to make the "average person" in the world wiser, rather than blame the SEO and bots...
Sounds quite ideal to me.
> hope some smart people will put some effort to make the "average person" in the world wiser
I'd rather hope for a future made by us and for us instead.
"Average" people just don't care about this stuff like we do. They don't care. I tried to get them to care, they refuse to care about all this stuff that we care about. That's fine, people like what they like and that's that but why should we care about their concerns then? We should not. And I do not.
Truth is I couldn't care less about such an "average" human being. Why is everything always about the "average" person? Why must all technology serve this mythical average human? Where is the technology that serves me? My programmer's computer system and network?
Isn't that why we all come to Hacker News?
People chase these "averages" because there's money in it. The money mostly comes from advertising consumer products to them. That's why advertising destroys everything.
Put it all in perspective
LLMs are only a deathblow, albeit a massive one, to a trend that's been 10 years in the making.
I feel it's time for another small internet for us, with blackjack and dancers. But this time let's agree not to make it friendly and accessible for everyone, alright?
and Slurm, please.
..anyway that's how I ended up with a Great Dane."
Feels well moderated too, by people who care about the place, and that's the core of a sustainable community. Care really matters.
On my own site, I routinely post lists of links to interesting articles. You don't have to rely on Twitter or other highly botted sources.
That most people do is itself revealing.
"More boring" is a defence against spammers, but not the only the one.
There's also: "Take commercial incentives of the running company out of the picture."
The most prominent example of that is Mastodon. It's software is opensourced, and it's most popular server instance: https://mastodon.social is run by a gGmbH non-profit. (It's hosting company runs it as a non-profit charity for the social good)
Since it's developed in the open without financial incentives muddying up the experience, no advertisements are added and there aren't any algorithmic rankings to be gamed.
And since it's also based on open source, it's easy to share content from other server instances (it's all ActivityPub protocol underneath), and it's also easy to block (defederate) server instances with trolls and other problematic users.
------------------------------
It's like how twitter was at the start, but better.
If you want to try it out, you can make an account on any server then follow some developers in your languages/libraries/tooling of your choice.
You'll also see what those maintainers are discussing in the open and get an idea of how your languages/libraries/tools are going to evolve in the next version, or even participate in their evolution.
Platforms like HN and Mastodon are great because they are small. They cater more towards a smaller, more technical community, which it isn't worth it to game with spam or whatnot because they're small and more aware of this kind of manipulation. Smaller "gems" in bigger platforms (think a small, old subreddit) can be good for the same reason.
I guess this advocates more for the small web, which I'm all for, but there's less money in that. I wonder what could practically what incentives could make the web smaller and more useful.
HN has 39 million comments after 17 years.
HN is small.
You only see who you follow, and there's no like/karma/upvotes algorithm.
Everything is sorted chronologically, and if anyone tries to "game" that by posting too much, they'd get unfollowed and/or banned.
Mastodon is a nurtured cultivated twitter.
Personally I follow the CSS/JS/TS community (for work), the gamedev community (for fun), and the space community (for passion)
Defense against spammers/bots is a tough problem though. Having great moderators and users who are savvy and intolerant of bad content probably helps but I think it would only go so far on a large site. HN might benefit from the relatively narrow appeal of its content in that regard.
HN is heavily moderated (censored) and people love to complain when that happens on sites like twitter.
I think proof of identity may soon become the only way to keep AI powered bots from trying to manipulate every forum out there.
And for those who cherish the ability to be anonymous, there is probably a market for a trusted middle-man/site that can verify that an account is being created by a biological person, but that doesn't provide the actual identity of that person to the site where he/she is setting up a new account. Kind of like CA providers do for ssl domains.
Community is hard. And requires moderation. And it's done here with great care. Don't worry about the spam, the fight is real but the problem is also very appreciated on the ground here by Dang and mods as you can tell. Keep it up. And meanwhile you can help by submitting quality content from regularly reliable sources, not upvoting dupes, contributing thoughtful discussion and keeping discussion focused/organized, as many of you do.
I'm curious what the end result will be but will be disappointed if the quality of HN decreases to be more in line with Reddit et al.
What makes HackerNews different from Reddit is that people I actually respect frequent here, real players at the game of life and whatnot (whatever that means), and I can't say the same about Reddit. If it gets too popular I can see that being jeopardized.
who knows.
Create a devils pact with the elite astroturfing people who no one seems to be bothered by or able to identify. Exchange data, and root out the "problem".
If you can't beat them, join them. The lesser evil is still perceived as a better experience after all. Our enemy his enemy is our friend.
Sucks, but is there an alternative?
I’m open to the idea of no new accounts if it got bad enough.
I recently switched to more actively using my account that has my name as the username, anticipating the growing problem of spam detection, and trying to make it more clear this account is not spam.
Also, identification is not something that is revocable. If someone steals and abuses your identity (though no fault of your own?), are you now banned from the internet?
there’s no identity system today that I would want HN to use.
maybe submissions only after reaching a certain number of points. that would reduce the number of people that can submit though.
another alternative would be to clearly identify which submissions come from posters with a high number of points. that would allow anyone to submit but make it easy to give posts by weak accounts a greater scrutiny.
you have to be 'approved' to post a topic, which you get by commenting on existing topics over time. TBH, I've never even bothered opening an account as I find it relaxing to be in 'read only' mode :-)
Huh. I'd heard that mentioned a bunch as a place with great conversational content, but had assumed (based on the crustacean naming and the `.rs` TLD) that it was Rust-focused - and, although I'm fascinated by and trying-to-learn Rust, I'm not yet at the level where I would be able to contribute to (or barely even learn much from) such a place.
But, having checked it out, it seems to in fact just be "Reddit, but more focused and with better moderation". Sounds delightful! Unfortunately I don't have enough of a social circle to be able to get an invite, but I'll enjoy reading in read-only mode. Thanks!
[1] https://www.kiwop.com/en/blog/parasite-seo-what-is-it-and-ho....
It is a preciously rare gem on today's web.
site:...
I do the latter and it works perfectly well, for example "apple vision pro hackernews" or "apple vision pro reddit".
(I myself prefer using the native HN search / duckduckgo bangs, because it can sort by date and shows the comment count for each thread).
I have noticed an increase in errors here in the past few months though, I wonder if this is the cause.
I have to actively exclude Quora to get a better answer.
The enshittification of the Web has truly begun.
A comment is the top result for "where was gta made"."
Not for me. Wikipedia is #1. I retrieved 100 results. None of them were HN comments.
"A comment from desdiv is ranking #1 for the question "Monaco in italian""
For me Wikipedia was again #1. HN comment was #2.
"Another article that ranks well for several keywords like "ruined my life" is GRAHAM PANTHER's article, and HN also ranks well for brand-related KW, such as every question about the cost of youtube tv, where Google favours the post from thunderbong."
For the search "cost of youtube tv", NerdWallet ranked #1. HN comment was #10.
It is baffling to me that people, who certainly know that results for the same query can vary according to a number of factors, e.g., geolocation, pretend every computer user performing some Google search query is getting identical results.
Ideally that is how search should work. But Google web search is not ideal. Far from it. It is advertising company search. The goal is not to return results in a consistent, transparent manner according to a relatively simple, well-known algorithm. It is to sell advertising services. The goal is not truth. It's money.
Does local search benefit sales of advertising services. Proponents of so-called "tech" companies will always try to justify the choices of the so-called "tech" company^1 as being for the benefit of the user. It could be half true.
To use DNS as an analaogy, EDNS0 client subnet (ECS) was another one of Google's magnificient "improvements" to the internet. Google fans would argue that ECS is 100% for the benefit of users. However it just so happens to be useful for advertising.^2 What a fortuitous coincidence.
1. The user does not choose. Choice is removed.
Click the 3 dots (kebab) beside individual search results and see if it was -
"Personalised for you"
This is the most likely answer.
[Edit] "where was gta made" does come back as "Not Personalised" in incognito mode and I see OP is in SEO.
I get a lot of results like this but all OPs examples seem "Not Personalised"