5,764 karma · joined June 23, 2008
First employee at RethinkDB (S09)
jdoliner@pachyderm.io medium.com/@jdoliner twitter.com/jdoliner github.com/jdoliner
[ my public key: https://keybase.io/jdoliner; my proof: https://keybase.io/jdoliner/sigs/ahN_pe7Uu2rmpbLvnm5anzLwsrErj7MrljfYZWQa9-0 ]
1. I give you a proof, you tell me if it's correct
2. I give you a theorem, you give me a correct proof
3. I give you nothing, you give me a theorem
1. is largely solved by modern LLMs and they took a big step toward 2. today with the Navier-Stokes proof. But they're definitely not there yet. It's unclear what progress is being made toward 3. for the time being that remains the realm of humans.
So I think maybe the guidelines should say something like:
HN readers appreciate research in comments that brings information relevant to the post. The best way to make such a comment is to find the information, summarize it in your own words that explain why it's relevant to the post and then link to the source if necessary. Adding "$AI said" or "Google said" generally makes your post worse.
---------
Also I asked ChatGPT and it said:
Short Answer
HN shouldn’t outright ban those comments, but it should culturally discourage them, the same way it discourages low-effort regurgitation, sensationalism, or unearned certainty. HN works when people bring their own insight, not when they paste the output of a stochastic parrot.
A rule probably isn’t needed. A norm is.
Now there's lots of variables that can be tweaked on this. So it's possible to get it to work. But there's a lot less room for error.
And often when the two are arguing it's tricky to tell which is which, because whether or not it does something isn't totally black and white, there's some things it can sometimes do, which you can argue either way about that being in its capabilities or not.
This of course now gets weird with LLMs because I doubt it can last as a signal of respect for very long when it just means you fed some bullet points to ChatGPT.
I think it's at least as plausible that this is part of the magic that makes it good. HN is sufficiently "on the margin" that they don't have to do things like placate advertisers with their moderation policies. The mods like dang, tomhow and pg mostly care about HN as users rather than owners.
> It would be a good thing for the world if HN was spun out as a non-profit and maintained long-term.
That sounds good in theory... in practice it might be the beginning of the end. Once there's a non-profit behind it the non-profit has a mission of its own. Although I'm actually not sure of the legal status of HN right now, maybe it's already something like that.
The biggest one seems to be a case of Goodhart's Law, leading to herding. Pollsters care a lot now about what their rating is in forecasting models, so they're reluctant to publish outlier results, those outlier results are very valuable for the models but are likely to get a pollster punished in the ratings next cycle.
Lots of changes to polling methods have been made due to polls underestimating Trump. Polls have become like mini models unto themselves. Due to their inability to poll a representative slice of the population they try to correct by adjusting their results to compensate for the difference between who they've polled and the likely makeup of the electorate. This makes sense in theory, but of course introduces a whole bunch of new variables that need to be tuned correctly.
On top of all this is the fact that the process is very high stakes and emotional with pollsters and modellers alike bringing their own political biases and only being able to resist pressure from political factions so much.
The analogy I kept coming back to watching election models during this last cycle was that it looked like an ML model that didn't have the data it needed to make good predictions and so was making the safest prediction it could make given what it did have. Basically getting stuck in this local minima at 50-50 that was least likely to be off by a lot.