HNHacker News
TopNewBestAskShowJobs

gwern

35,358 karma · joined April 8, 2009

Site: https://gwern.net/

About me: https://gwern.net/me

What I've written recently: https://gwern.net/changelog

submissionscomments
gwern··on A font with built-in TeX syntax highlighting
Note: this is different from https://blog.glyphdrawing.club/font-with-built-in-syntax-hig... https://news.ycombinator.com/item?id=41245159 which recently resurfaced, and is cited as prior art.
gwern··on Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB
So if it's not using attention and it processes the entire input into an embedding to process in one go, I guess this is neither a Transformer nor a RNN but just a MLP?
gwern··on Scientists edited genes in a living person and saved his life
Note: from May 2025. https://gwern.net/doc/genetics/editing/2025-musunuru.pdf
gwern··on Grok and the Naked King: The Ultimate Argument Against AI Alignment
Isn't even thoughtful either.

> The question was never “how do we align AI with human values?” The question was always “which humans get to define those values?” Grok answered that question: the ones with the most money.

Grok is routinely misaligned with Elon, as the article points out in its intro! You don't need to order your engineers to keep fixing what isn't broken...

gwern··on Coarse is better
> I would also welcome someone writing a short takedown where they fix the prompts and get better-than-2022 results from nbp

NBP (and the new ChatGPT generator) are integrated with LLMs to various degrees, so seems like the obvious starting point is a reverse approach: ask them to describe the old images which has the esthetics that Fernando Borretti likes, and start generating from those prompts. If you can recover the old images, then it was just a prompting issue. ("Sampling can show the presence of knowledge but not the absence.") If you can't even with their own 'native' descriptions, then that points to mode-collapse (especially all of the 'esthetic tuning' like DPO everyone does now) as being the biggest problem.

gwern··on Inca Stone Masonry
Penn & Teller formulation: https://www.smithsonianmag.com/arts-culture/teller-reveals-h...
gwern··on Detailed balance in large language model-driven agents
There's a vein of research which interprets self-attention as a kind of gradient descent and says that LLMs have essentially pre-solved indefinitely large 'families' or 'classes' of tasks, and the 'learning' they do at runtime is simply gradient descent (possibly Newton) using the 'observations' to figure out which pre-solved instance they are now encountering; this explains why they fail in such strange ways, especially in agentic scenarios - because if the true task is not inside those pre-learned classes, no amount of additional descent can find it after you've found the 'closest' pre-learned task to the true task. (Some links: https://gwern.net/doc/ai/nn/transformer/attention/meta-desce... )

I wonder if this can be interpreted as consistent with that 'meta-learned descent' PoV? If the system is fixed and is just cycling through fixed strategies, that is what you'd expect from that: the descent will thrash around the nearest pre-learned tasks but won't change the overall system or create new solved tasks.

gwern··on NOAA deploys new generation of AI-driven global weather models
There's also the efficiency argument from new capability: even a tiny bit better weather forecast is highly economically valuable (and saves a lot of wasted energy) if it means that 1 city doesn't have to evacuate because of an erroneous hurricane forecast, say. But how much would it cost to do that with the rivals? I don't know but I would guess quite a lot.

And one of the biggest ironies of AI scaling is that where scaling succeeds the most in improving efficiency, we realize it the least, because we don't even think of it as an option. An example: a Transformer (or RNN) is not the only way to predict text. We have scaling laws for n-grams and text perplexity (most famously, from Jeff Dean et al at Google back in the 2000s), so you can actually ask the question, 'how much would I have to scale up n-grams to achieve the necessary perplexity for a useful code writer competitive with Claude Code, say?' This is a perfectly reasonable, well-defined question, as high-order n-grams could in theory write code without enough data and big enough lookup tables, and so it can be answered. The answer will look something like 'if we turned the whole earth into computronium, it still wouldn't be remotely enough'. The efficiency ratio is not 10:1 or 100:1 but closer to ∞:1. The efficiency gain is so big no one even thinks of it as an efficiency gain, because you just couldn't do it before using AI! You would have humans do it, or not do it at all.

gwern··on Prompt caching for cheaper LLM tokens
No, the search space is tiny: you can just attack 1 BPE at a time! Stuff like password guessing is almost trivial when you get to do a timing attack on each successive character. So that lets you quickly exfiltrate arbitrary numbers of prompts, especially if you have any idea what you are looking for. (Note that a lot of prompts are already public information, or you can already exfiltrate prompts quite easily from services and start attacking from there...)
gwern··on AI generated font using Nano Banana
While we're at it: my own work 2 years ago in creating an entire workflow for turning Midjourney or DALL-E dropcaps into attractive, lightweight, easy-to-create dropcaps for web pages: https://gwern.net/dropcap We use it for the cat, Gene Wolfe, and holiday pages.
gwern··on Gwern's "Stem Humor" Directory
No. I think the need for adversarial losses in order to distill diffusion models into one-step forward passes has provided additional evidence that GANs were much more viable than diffusimaxis loudly insisted.

(Although I'm not really current on where image generation is these days or who is using GAN-like approaches under the hood or what are the current theoretical understandings of GAN vs AR vs diffusion, so if you have some specific reason I should have "caved", feel free to mention it - I may well just be unaware of it.)

gwern··on AsciiMath
This works surprisingly well. If you look into enough dark corners of Unicode, it turns out that you can do a shocking amount of typography, going far beyond the obvious italics and bolds: https://gwern.net/utext

In fact, I found that writing as much math as possible in Unicode makes for the best HTML reading experience: it's fast, simple, and looks more natural (avoids font inconsistency and line-height jagginess, among other things). https://gwern.net/design-graveyard#mathjax

And if you find writing Unicode yourself a pain, you can just ask a LLM to translate from LaTeX to Unicode! https://github.com/gwern/gwern.net/blob/master/build/latex2u...

gwern··on Antislop: A framework for eliminating repetitive patterns in language models
> but I think fixing the verbal tics is still important and useful.

I don't. I think they're useful for flagging the existence of mode-collapse and also providing convenient tracers for AI-written prose. Erasing only the verbal tics with the equivalent of 's/ - /; /g' (look ma! no more 4o em dashes!) is about the worst solution you could come up with and if adopted would lead to a kind of global gaslighting. The equivalent of a vaccine for COVID which only suppresses coughing but doesn't change R, or fixing a compiler warning by disabling the check.

If you wanted to do useful research here, you'd be doing the opposite. You'd be figuring out how to make the verbal expressions even more sensitive to the underlying mode-collapse, to help research into fixing it and raising awareness. (This would be useful even on the released models, to more precisely quantify their overall mode-collapse, which is poorly captured by existing creative writing benchmarks, I think, and one reason I've had a hard time believing things like Eqbench rankings.)

gwern··on Antislop: A framework for eliminating repetitive patterns in language models
This is the epitome of patching symptoms rather than treating the disease. Even if you suppress the obvious syntactic slop like 'it's not X but Y', you have no reason to believe you've fixed mode-collapse on higher more important levels like semantics and creativity. (For example, Claude LLMs have always struck me as mode-collapsed on a semantic level: they don't have the blatant verbal tics of 4o but somehow they still 'go in circles'.) Which will potentially severely hinder the truly high-value applications of LLMs to creative applications like frontier research. To the extent that this succeeds in hiding the brain damage in contemporary LLMs, it arguably is a cure worse than the disease.
gwern··on Designing software for things that rot
You're correct, but when the worst the ChatGPTisms get is turns of phrases like "LeetCode youth finally paid off: turns out all those "rebalance a binary search tree" problems were preparing me for salami, not FAANG interviews." or "Designing software for things that rot means optimising for variance, memory, and timing–not perfection. It turns out the hardest part of software isn't keeping things alive. It's knowing when to let them age.", then I'm inclined to forgive it compared to how many far more egregious offenders are at the top of HN these days. This is a rather mild use of ChatGPT for copyediting, and at least I feel like I can trust OP to factcheck everything and not put in any confabulations.
gwern··on Who invented deep residual learning?
My point is more that Schmidhuber is saying that the gates or the initialization are the innovation solely because they produce well-behaved gradients, which is why Hochreiter's 1991 thing is where he starts and nothing before that counts. But it's not clear to me why we should define it like that when you can solve the gradient misbehavior other ways, which is why https://gwern.net/doc/ai/nn/fully-connected/1988-lang.pdf#pa... works and doesn't diverge: if I'm understanding them right, they did warmup, so the gradients don't explode or vanish. So why doesn't that count? They have shortcut layers and a solution to exploding/vanishing gradients and it works to solve their problem. Is it literally 'well, you didn't use a gate neuron or fancy initialization to train your shortcuts stably, therefore it doesn't count'? Such an argument seems carefully tailored to exclude all prior work...
gwern··on Who invented deep residual learning?
> Note again that a residual connection is not just an arbitrary shortcut connection or skip connection (e.g., 1988)[LA88][SEG1-3] from one layer to another! No, its weight must be 1.0, like in the 1997 LSTM, or in the 1999 initialized LSTM, or the initialized Highway Net, or the ResNet. If the weight had some other arbitrary real value far from 1.0, then the vanishing/exploding gradient problem[VAN1] would raise its ugly head, unless it was under control by an initially open gate that learns when to keep or temporarily remove the connection's residual property, like in the 1999 initialized LSTM, or the initialized Highway Net.

After reading Lang & Witbrock 1988 https://gwern.net/doc/ai/nn/fully-connected/1988-lang.pdf I'm not sure how convincing I find this explanation.

gwern··on Show HN: I invented a new generative model and got accepted to ICLR
This apparently doesn't apply here, but in fact, pixels can be generated independently of each other. There are architectures where you can generate an arbitrary pixel or element of the image without generating the others; they are just implicit. See NeRFs or 'single-pixel GANs' or MAEs: eg https://arxiv.org/abs/2003.08934 https://arxiv.org/abs/2011.13775 https://arxiv.org/abs/2401.14391

Why is this possible? I tend to think of it as reflecting the ability to 'memorize' all possible data, and the independent generation is just when you 'remember' a specific part of a memory. The latent space is a Platonic object which doesn't change, so why should your generative process for materializing any specific point in the latent space have to? It's not surprising if you could generate arbitrary points from a function like 'y = mx + b' without generating every other point, right? It's just an atemporal mathematical object. Similarly with 'generating images from a random seed'. They too are just (complicated) functions mapping one number to another number.

(You might wonder if this is limited to images? It is not. In fact, you can generate even natural language like this to some degree: https://github.com/ethan-w-roland/AUNN based on my proposal for taking the 'independent generation' idea to a pathological extreme: https://gwern.net/aunn )

gwern··on Claude Sonnet 4.5
Requires a login?
gwern··on Investigating a Forged PDF
I think OP is an instance that proves the point

this is a lazy, clumsy editing attempt, done through a document registration service which exists to prevent exactly this and yet, you have to be an experienced nerd (eg https://en.wikipedia.org/wiki/Matthew_Garrett has a doctorate and decades of software development experience) who will jump through a bunch of hoops to even begin to build a case beyond he-said-she-said. And he still doesn't have a settlement or criminal conviction in hand, so he's not even half done... Or look at the extensive forensics in the Craig Wright case just to establish simple things like that they were edited or backdated to a legally acceptable level.

Meanwhile, the original PDF edit in question took maybe 5 minutes with entry-level PDF tools.

gwern··on Just let me select text
One of the most surprising Gwern.net bug reports was from a compulsive highlighter who noted that the skip-ink implementation (which uses the old text-shadow trick, because frustratingly, the recently standardized skip-ink CSS still manages to fail at its only job and it looks awful) looked bad because of how browsers handle shadows and highlighting.

We had known about that (and it can't be fixed because browsers don't let you control the highlighting), but we had never imagined it'd be a problem because you'd only see it briefly when once in a while copy-pasting some text for a quote - right? I mean, why else would anyone be highlighting text? You'd only highlight small bits of text you had already read, so if it looked bad in places, that was fine, surely.

(Narrator: "It was not fine.")

Just another instance of Hyrum's law, I guess...

We decided to WONTFIX that because we can't easily fix that without making it uglier for users who don't abuse highlighting and are reading normally, which is almost everyone else.

gwern··on Game over for pure LLMs. Even Rich Sutton has gotten off the bus
Demonstrating that Rich Sutton was never really on the 'LLM bus' in the first place. Note the remarkable absence from the essay of language models & large language models from that essay despite BERT and GPT-2 and 'unreasonable effectiveness of data' etc. He only briefly mentions speech recognition. (Note also Sutton's general absence from LLM research, the Edmund Plan or switch from DeepMind to Keen Technologies as DeepMind was forced into LLM-centric research, and his published research since 2019's emphasis on small models and trying to fix their pathologies like catastrophic forgetting.)

> The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.

You could easily seem most LLM work as a dead end because it is about 'building knowledge into your agents' (eg. by paying data labelers billions of dollars total to supplement your scrapes), and not about 'search' (still a major open problem for LLMs - o1-style serial reasoning traces are obviously inadequate) or 'learning' (LLMs depend so heavily on the knowledge already encoded in so much data for them).

gwern··on AGI is an engineering problem, not a model training problem
I don't have any particularly canonical reference I'd cite here, but self-preference bias in LLMs is well-established. (Just search Arxiv.)
gwern··on AGI is an engineering problem, not a model training problem
They can, but they are known to have a self-favoring bias, and in this case, the error is so easily identified that it raises the question of why GPT-5 would both come up with it & preserve it when it can so easily identify it; while if that was part of OP's original inputs (whatever those were) it is much less surprising (because it is a common human error and mindlessly parroted in a lot of the 'scaling has hit a wall' human journalism).
gwern··on AGI is an engineering problem, not a model training problem
It is lacking in URLs or references. (The systematic error in the self-reference blog post URLs is also suspicious: outdated system prompt? If nothing else, shows the human involved is sloppy when every link is broken.) The assertions are broadly cliche and truisms, and the solutions are trendy buzzwords from a year ago or more (consistent with knowledge cutoffs and emphasizing mainstream sources/opinions). The tricolon and unordered bolded triplet lists are ChatGPT. The em dashes (which you should not need to be told about at this point) and it's-not-x-but-y formulation are extremely blatant, if not 4o-level, and lacking emoji or hyperbolic language; hence, it's probably GPT-5. (Sub-GPT-5 ChatGPTs would also generally balk at talking about a 'GPT-5' because they think it doesn't exist yet.) I don't know if it was 100% GPT-5-written, but I do note that when I try the intro thesis paragraph on GPT-5-Pro, it dislikes it, and identifies several stupid assertions (eg. the claim that power law scaling has now hit 'diminishing returns', which is meaningless because all log or power laws always have diminishing returns), so probably not completely-GPT-5-written (or least, sub-Pro).
gwern··on AGI is an engineering problem, not a model training problem
It is especially not obvious because this was written using ChatGPT-5. One appreciates the (deliberate?) irony, at least. (Or at least, surely if they had asymptoted, OP should've been able to write this upvoted HN article with an old GPT-4, say...)
gwern··on Everything is correlated (2014–23)
https://gwern.net/doc/statistics/causality/1926-yule.pdf
gwern··on Everything is correlated (2014–23)
That is unintentional and a bug in the dark-mode.

For dark-mode, we rely on https://invertornot.com/ to dynamically decide whether to fade or negate/invert. (Background: https://gwern.net/invertornot ) The service uses a small NN and is not always correct, as in these cases. Sorry.

I have filed them as errors with InvertOrNot, and will manually set them to invert.

gwern··on The Day Novartis Chose Discovery
The final section pounding the desk about how terrible ending the program was seems like it is oddly at variance with all the evidence OP had just laid out about how the program wasn't working well anymore and so wasn't actually financially a good idea. It's weird to quote a bunch of things like studies showing that 'internal R&D spending works worse than external for ROI' and then write a big moralizing sermonizing conclusion about how ending internal R&D is bad for profits and how terrible it is there's no 'patient capital' (capital which was plenty available before - what's the theory, investors stopped liking making money? insurance companies with century-long investment horizons ceased to exist? etc).
gwern··on The Inkhaven Blogging Residency
The mountain gave birth to a mouse.
← PreviousPage 9 of 34Next →