35,358 karma · joined April 8, 2009
About me: https://gwern.net/me
What I've written recently: https://gwern.net/changelog
> The question was never “how do we align AI with human values?” The question was always “which humans get to define those values?” Grok answered that question: the ones with the most money.
Grok is routinely misaligned with Elon, as the article points out in its intro! You don't need to order your engineers to keep fixing what isn't broken...
NBP (and the new ChatGPT generator) are integrated with LLMs to various degrees, so seems like the obvious starting point is a reverse approach: ask them to describe the old images which has the esthetics that Fernando Borretti likes, and start generating from those prompts. If you can recover the old images, then it was just a prompting issue. ("Sampling can show the presence of knowledge but not the absence.") If you can't even with their own 'native' descriptions, then that points to mode-collapse (especially all of the 'esthetic tuning' like DPO everyone does now) as being the biggest problem.
I wonder if this can be interpreted as consistent with that 'meta-learned descent' PoV? If the system is fixed and is just cycling through fixed strategies, that is what you'd expect from that: the descent will thrash around the nearest pre-learned tasks but won't change the overall system or create new solved tasks.
And one of the biggest ironies of AI scaling is that where scaling succeeds the most in improving efficiency, we realize it the least, because we don't even think of it as an option. An example: a Transformer (or RNN) is not the only way to predict text. We have scaling laws for n-grams and text perplexity (most famously, from Jeff Dean et al at Google back in the 2000s), so you can actually ask the question, 'how much would I have to scale up n-grams to achieve the necessary perplexity for a useful code writer competitive with Claude Code, say?' This is a perfectly reasonable, well-defined question, as high-order n-grams could in theory write code without enough data and big enough lookup tables, and so it can be answered. The answer will look something like 'if we turned the whole earth into computronium, it still wouldn't be remotely enough'. The efficiency ratio is not 10:1 or 100:1 but closer to ∞:1. The efficiency gain is so big no one even thinks of it as an efficiency gain, because you just couldn't do it before using AI! You would have humans do it, or not do it at all.
(Although I'm not really current on where image generation is these days or who is using GAN-like approaches under the hood or what are the current theoretical understandings of GAN vs AR vs diffusion, so if you have some specific reason I should have "caved", feel free to mention it - I may well just be unaware of it.)
In fact, I found that writing as much math as possible in Unicode makes for the best HTML reading experience: it's fast, simple, and looks more natural (avoids font inconsistency and line-height jagginess, among other things). https://gwern.net/design-graveyard#mathjax
And if you find writing Unicode yourself a pain, you can just ask a LLM to translate from LaTeX to Unicode! https://github.com/gwern/gwern.net/blob/master/build/latex2u...
I don't. I think they're useful for flagging the existence of mode-collapse and also providing convenient tracers for AI-written prose. Erasing only the verbal tics with the equivalent of 's/ - /; /g' (look ma! no more 4o em dashes!) is about the worst solution you could come up with and if adopted would lead to a kind of global gaslighting. The equivalent of a vaccine for COVID which only suppresses coughing but doesn't change R, or fixing a compiler warning by disabling the check.
If you wanted to do useful research here, you'd be doing the opposite. You'd be figuring out how to make the verbal expressions even more sensitive to the underlying mode-collapse, to help research into fixing it and raising awareness. (This would be useful even on the released models, to more precisely quantify their overall mode-collapse, which is poorly captured by existing creative writing benchmarks, I think, and one reason I've had a hard time believing things like Eqbench rankings.)
After reading Lang & Witbrock 1988 https://gwern.net/doc/ai/nn/fully-connected/1988-lang.pdf I'm not sure how convincing I find this explanation.
Why is this possible? I tend to think of it as reflecting the ability to 'memorize' all possible data, and the independent generation is just when you 'remember' a specific part of a memory. The latent space is a Platonic object which doesn't change, so why should your generative process for materializing any specific point in the latent space have to? It's not surprising if you could generate arbitrary points from a function like 'y = mx + b' without generating every other point, right? It's just an atemporal mathematical object. Similarly with 'generating images from a random seed'. They too are just (complicated) functions mapping one number to another number.
(You might wonder if this is limited to images? It is not. In fact, you can generate even natural language like this to some degree: https://github.com/ethan-w-roland/AUNN based on my proposal for taking the 'independent generation' idea to a pathological extreme: https://gwern.net/aunn )
this is a lazy, clumsy editing attempt, done through a document registration service which exists to prevent exactly this and yet, you have to be an experienced nerd (eg https://en.wikipedia.org/wiki/Matthew_Garrett has a doctorate and decades of software development experience) who will jump through a bunch of hoops to even begin to build a case beyond he-said-she-said. And he still doesn't have a settlement or criminal conviction in hand, so he's not even half done... Or look at the extensive forensics in the Craig Wright case just to establish simple things like that they were edited or backdated to a legally acceptable level.
Meanwhile, the original PDF edit in question took maybe 5 minutes with entry-level PDF tools.
We had known about that (and it can't be fixed because browsers don't let you control the highlighting), but we had never imagined it'd be a problem because you'd only see it briefly when once in a while copy-pasting some text for a quote - right? I mean, why else would anyone be highlighting text? You'd only highlight small bits of text you had already read, so if it looked bad in places, that was fine, surely.
(Narrator: "It was not fine.")
Just another instance of Hyrum's law, I guess...
We decided to WONTFIX that because we can't easily fix that without making it uglier for users who don't abuse highlighting and are reading normally, which is almost everyone else.
> The bitter lesson is based on the historical observations that 1) AI researchers have often tried to build knowledge into their agents, 2) this always helps in the short term, and is personally satisfying to the researcher, but 3) in the long run it plateaus and even inhibits further progress, and 4) breakthrough progress eventually arrives by an opposing approach based on scaling computation by search and learning. The eventual success is tinged with bitterness, and often incompletely digested, because it is success over a favored, human-centric approach.
You could easily seem most LLM work as a dead end because it is about 'building knowledge into your agents' (eg. by paying data labelers billions of dollars total to supplement your scrapes), and not about 'search' (still a major open problem for LLMs - o1-style serial reasoning traces are obviously inadequate) or 'learning' (LLMs depend so heavily on the knowledge already encoded in so much data for them).
For dark-mode, we rely on https://invertornot.com/ to dynamically decide whether to fade or negate/invert. (Background: https://gwern.net/invertornot ) The service uses a small NN and is not always correct, as in these cases. Sorry.
I have filed them as errors with InvertOrNot, and will manually set them to invert.