HNHacker News
TopNewBestAskShowJobs

gwern

35,358 karma · joined April 8, 2009

Site: https://gwern.net/

About me: https://gwern.net/me

What I've written recently: https://gwern.net/changelog

submissionscomments
gwern··on Generative AI and Wikipedia editing: What we learned in 2025
Thanks for the LLM comment, but that's dumb. If the problem really was as bad with humans (it obviously is not), then OP wouldn't've happened:

> For most of the articles Pangram flagged as written by GenAI, nearly every cited sentence in the article failed verification.

gwern··on Deterministic Governance: mechanical exclusion / bit-identical
I don't get it. Embeddings don't prioritize, or even necessarily encode, truth value as a dimension. And even if they did, if you simply accept based on some hyperparameter of distance, it sounds like this procedure just leaves you vulnerable to problems like salami-slicing where you reach 'the sky is green' (which after all, it is sometimes) by multiple steps just below the tolerance.
gwern··on Deterministic Governance: mechanical exclusion / bit-identical
OK, this is AI slop ("fracture" alone gives it away). But maybe there's still something of value here? Can you explain it in actual human terms, give a real example, and explain what you did to test this and why I shouldn't flag this like I did https://news.ycombinator.com/item?id=46701114 ?
gwern··on Self Driving Car Insurance
A discount they get to set on a subset of miles of their choice may just be a marketing expense for an insurance startup which makes losses and relies on VC capital and needs growth: https://en.wikipedia.org/wiki/Lemonade,_Inc.#2015%E2%80%9320... I was impressed by this until I looked Lemonade up.
gwern··on Recent discoveries on the acquisition of the highest levels of human performance
(To stop being lazy, the 2020 reference is: https://news.ycombinator.com/item?id=25426329 )
gwern··on Trinity large: An open 400B sparse MoE model
> E.g. gemini-3-pro tops the lmarena text chart today at 1488 vs 1346 for gpt-4o-2024-05-13. That's a win rate of 70% (where 50% is equal chance of winning) over 1.5 years. Meanwhile, even the open weights stuff OpenAI gave away last summer scores between the two.

Why do you care about LM Arena? It has so many problems, and the fact that no one would suggest using GPT-4o for doing math or coding right now, or much of anything, should tell you that a 'win rate of 70%' does not mean whatever it looks like it means. (Does GPT-4o solve roughly as many Erdos questions as gemini-3-pro...? Can you write roughly as good poetry?)

gwern··on The behavioral cost of personalized pricing
Hm... I think I would predict that it would depend on speed because querying prices reveals information, but not accepting also reveals information, so I would expect that it would go up initially due to an expected shortage, but then when it becomes clear that no one is accepting, it reverts to normal or lower, because that suggests low urgency or willingness-to-pay compared to what the models had forecast for that exact location & time, so they have to be offered lower prices to accept at all.

(This is the flipside of personalized pricing: it's also lower prices, as noted in some of the anecdotes, although he mostly emphasizes the potential for higher prices.)

gwern··on The behavioral cost of personalized pricing
Well, whichever one checked first, the point still holds. Checking is not a free operation and you should expect it to have consequences and it to affect Uber's demand forecasts etc., and affect other price quotes. You do not have independence here and so the comparison is not as meaningful as it seems. As the Roman wit said, "when two people do the same thing, it's not the same thing."

(The right way to do this is to randomize multiple independent occasions - wait until one of you was about to call an Uber, immediately flip a coin to decide who does, each time checking you or your wife's Uber, and never both, and compare the long-run average.)

> with an apologetic confirmation dialog, possibly only after some holding period has timed out

Right. I've seen the same thing myself. They would prefer not to apologize to the customer because they changed the price, because it is in fact annoying and a bad customer experience. So the prices are surely carefully set in many ways with an eye towards not changing as much as possible.

> This enables people to construct a plausible story to explain any price.

Indeed. So you should mention one of the most plausible stories if you're going to list a bunch of them.

gwern··on Any application that can be written in a system language, eventually will be
Not going far enough - why would applications be written in either 'systems languages' or 'agent languages' if you have superintelligence too cheap to meter and you will amortize the costs over more than, say, a few days? Just write in raw assembler from a domain-specific design hyperoptimized for solely the task, the way Donald Knuth on steroids would.
gwern··on The behavioral cost of personalized pricing
> Who knows why? I’m usually more willing to spend than she is, and I bet that's represented on my user profile. I was paying with a gift card, which surely contributes. Maybe it was a price scraping update, comparison shopping detection, or a system that explores “face-in-the-door” high prices before backing down. From the outside, no one really knows.

The most obvious possibility omitted is that your wife got the first, easy, cheap car and then Uber had to quote you a higher price to get a second car. Cars don't fall from the sky; if two people successively ask for bids, how else could it work? What if the app quoted you both the cheap price for the only car within X blocks, and you bought it before she did? Is it suddenly going to go 'oops sorry, changed my mind, it now costs twice as much'? Sounds like a very bad experience to me! More sensible to give the first person a low quote and then when - unexpected and unpredictably - someone requests something similar, quote them the higher price reflecting the sudden local micro-shortage.

gwern··on Recent discoveries on the acquisition of the highest levels of human performance
> It's like how software engineers who are at Google are worse if they're better competitive programmers.

That's not true. It didn't replicate and Norvig has said as much somewhere on HN, IIRC.

(I also agree with the other criticisms that this 'old vs young' setup in OP is obviously at least partially, and perhaps entirely, regression to the mean and Berkson.)

gwern··on Libbbf: Bound Book Format, A high-performance container for comics and manga
> I mean, its open source so people can create benchmark and independently verify if the AI was wrong and then have the claims be passed to the author.

Thank you for volunteering. I look forward to your results.

gwern··on Libbbf: Bound Book Format, A high-performance container for comics and manga
It makes no sense because it's some degree of AI slop: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_...

Note that he doesn't quite say, when asked pointblank how much AI he used in his erroneous microbenchmarking, that he didn't use AI: https://reddit.com/r/selfhosted/comments/1qi64pr/i_got_into_...

Which explains all of it.

Kudos to /u/teraflop, for having infinitely more patience with this than I would.

gwern··on LLM poetry and the "greatness" question: Experiments by Gwern and Mercor
Also of interest: https://dbohdan.com/kaur
gwern··on TimeCapsuleLLM: LLM trained only on data from 1800-1875
There's also severe selection effects: what documents have been preserved, printed, and scanned because they turned out to be on the right track towards relativity?
gwern··on Show HN: What if AI agents had Zodiac personalities?
I think my take away is that you are seeing mostly mode-collapse here. There is a high consistency across all of the supposedly different personalities (higher than the naive count would indicate - remember the stochastic nature of responses will inflate the number of 'different' responses, since OP doesn't say anything about sampling a large number of times to get the true response).
gwern··on Intellectual Junkyards
Writing or tweaking plugins is great, but it's not a paradigm shift (and risks a lot more toil because now you have to be your own PM or deal with patches/merges, on top of being a reference librarian and copyeditor etc). I feel like if you have a quasi-superintelligence in a box which can run your PKM for you, and you were designing from the ground up with this in mind, that Claude Code is only going to et much better & cheaper, you would not be settling for 'write or modify an Obsidian plugin'. You would get something much different. But 'write a plugin' is basically at 'horseless carriage' level for me.

What I have in mind is something far more radical. There's an idea I am calling 'log-only writing' where you stop editing or rearranging your notes at all, and you switch to pure note taking and stream of conscious braindumping, and you simply have the LLM 'compile' your entire history down into whatever specific artifact you need on demand - whether that's a web of flashcards or a blog post or a long essay or whatever. See https://gwern.net/blog/2024/rss + https://gwern.net/nenex , combined with the LLM reasoning and brainstorming 'offline' using the prompts illustrated by my poems.

gwern··on Intellectual Junkyards
No, but I haven't been following the space. (I suspect that with Claude Code-level coding agents, you should be able to do something amazing that thoroughly obsoletes Obsidian/Roam/org-mode, but I don't actually know of anything.)

I've been focused on creative writing, with poetry as my test case, to see what the bottlenecks are to truly amplifying myself through LLMs (as opposed to helping my boss automate away my job or spamming the Internet more efficiently).

I find that frontier LLMs are now there and now I can prompt for genuinely good poetry with LLMs. See https://hollisrobbinsanecdotal.substack.com/p/llm-poetry-and... / https://gwern.net/fiction/lab-animals and https://gwern.net/blog/2025/better-llm-writing

So maybe this year I can turn some attention back to PKMs and Quantified Self stuff...

gwern··on Intellectual Junkyards
Yes, and PKMs in general. Like labeling your emails by topic in Gmail. The problem is that the 'toil' keeps piling up, while the value gained is increasingly hard to see.

I have a little rant about it - "‘Tools for thought’ winds up being a lie: there’s tools, but not much additional thought." https://gwern.net/blog/2024/tools-for-thought-failure https://www.lesswrong.com/posts/CoqFpaorNHsWxRzvz/what-comes...

(My answer, of course, is that almost all of this scutwork is well within the capabilities of a frontier LLM today. We just need to apply them.)

gwern··on The Tyranny of the Complainers
The conclusion here seems largely unjustified by the data and indeed is difficult to relate to simple distributions or statistics:

> Increasingly, public institutions seem to exist to manage the obsessions of a tiny number of neurotic—and possibly malicious—complainers.

Why would anyone complain about airport noise when it is ~100% guaranteed to do them no good, and almost all the benefits go to everyone else even if it somehow did anything? Just thinking like an economist here... (Indeed, if a large fraction of locals did complain about something like airport noise, that would itself be highly suspicious to me - as it would indicate an organized campaign or an issue which has become politicized in some way and is now a pretext for something else entirely like a culture war.)

And if there is something I've learned about design and problems, it's that you can have a huge problem, and you are lucky if even 1% will ever tell you.

Your website could be down, and if even 1 person takes the risk of going out of their way to tell you, you should thank your lucky stars that you have such proactive, public-spirited readers!

See also: "Theory of the Nudnik: The Future of Consumer Activism and What We Can Do to Stop It" https://gwern.net/doc/economics/2020-arbel.pdf , Arbel & Shapira 2020; https://pointersgonewild.com/2019/11/02/they-might-never-tel... (commentary: https://gwern.net/ref/chevalier-boisvert-2019); https://en.wikipedia.org/wiki/1%25_rule

gwern··on On the slow death of scaling
> is not related to predicting what is possible with greater scale.

If they can't predict new emergence, then 'explaining' old emergence by post hoc prediction with bizarre newly-invented metrics would seem to be irrelevant and just epicycles. You can always bend a line as you wish in curve-fitting by adding some parameters.

gwern··on On the slow death of scaling
> I thought people recognized that they don't appear out of nowhere.

I don't think that paper is widely accepted. Have you seen the authors of that paper, or anyone else, use it to successfully predict (rather than postdict) anything?

gwern··on Shipping at Inference-Speed
1. You put a lot of time into an intern or a junior too. 2. I didn't say 'paid', I said, 'fully loaded [total] cost'. The total cost of them goes far beyond their mere salary - the search process like all of the interviews for all candidates, onboarding, HR, taxes etc.
gwern··on Shipping at Inference-Speed
> So he's spent $51k on tokens in 3 months to build what exactly? Tools to enable you to spend more money on tokens?

Sounds like that's more than a junior or an intern that would have cost twice as much in fully loaded cost.

gwern··on enclose.horse
Or the cherries could be a delicious pastry or PBJ-like treat: _Collect Horse Buttery Stable_...
gwern··on The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (2018)
(Cerebras is real hardware.)
gwern··on The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks (2018)
Could you explain why you think that? I'm looking at the lottery ticket section and it seems like he doesn't disown it; the reason he gives, via Abhinav, for not pursuing it at his commercial job is just that that kind of sparsity is not hardware friendly (except with Cerebras). "It doesn't provide a speedup for normal commercial workloads on normal commercial GPUs and that's why I'm not following it up at my commercial job and don't want to talk about it" seems pretty far from "disowning the lottery ticket hypothesis [as wrong or false]".
gwern··on Total monthly number of StackOverflow questions over time
> which while describes a curve that shows the same behavior, it's more "accurate" of the actual post creation.

I would say that this graph looks a lot more extreme, actually!

gwern··on A website to destroy all websites
I am guessing you are on mobile and are referring to the togglebar 'demo mode', illustrating its existence.

I wish we didn't have that, but we have to. That demo of the theme togglebar exists to educate users, because we found that a lot of complaints came from people who were blind to the gear icon and were unaware that the control they wanted already existed. (Which is regrettable but understandable, because almost all websites provide useless controls or visual spam, in an example of 'why we can't have nice things'.)

Obviously, it's disabled on subsequent page loads. (One of a number of parts of the UI/UX we streamline using 'demo-mode', as we try to thread the Scylla & Charybdis of a cluttered but explicit UI vs a clean newbie-unfriendly UI.)

gwern··on Fighting Fire with Fire: Scalable Oral Exams
To clarify the point here for people who didn't read OP: the oral exams here are customized and tailored to the student's individual unique project, that's the point and why they are not written:

> In our new "AI/ML Product Management" class, the "pre-case" submissions (short assignments meant to prepare students for class discussion) were looking suspiciously good. Not "strong student" good. More like "this reads like a McKinsey memo that went through three rounds of editing," good...Many students who had submitted thoughtful, well-structured work could not explain basic choices in their own submission after two follow-up questions. Some could not participate at all...Oral exams are a natural response. They force real-time reasoning, application to novel prompts, and defense of actual decisions. The problem? Oral exams are a logistical nightmare. You cannot run them for a large class without turning the final exam period into a month-long hostage situation.

Written exams do not do the same thing. You can't say 'just do a written exam'. So sure, the students may prefer them, but so what? That's apples and oranges.

← PreviousPage 8 of 34Next →