gwern

Insider trading is misappropriating knowledge. Not really misappropriation here.

Maybe you're looking for 'moral hazard'?

(Pangram commentary: https://x.com/max_spero_/status/2028620466995220568 https://x.com/max_spero_/status/2028689219112034436 )

People are making a big deal of the fact that there's multiple prompts or interactions, but so what? That's true of source code too! The final commit doesn't reflect every last keystroke or thing you ever said while working on it; you may have had a long conversation with your coworkers, but then you kept only the summary part that mattered. It is the final, distilled, comprehensive patch.

So the requirement should be that there is a prompt, or set of prompts, plus documentation, which distill out everything learned or done in a single master prompt which then suffices to build a similarly capable codebase. ('Reproducibility' in the sense of replication, getting roughly the same result, not 'reproducibility' in the sense of bit-identical builds. (This is just 'compaction' on steroids, and something I generally do not have problem doing with my own LLM creative writing, no matter how many iterations or conversations it took to get to the final idea.)

If it s not possible to create such a thing for a codebase, even with the vibecoded artifact at hand, then that raises an awful lot of questions about the quality or what went wrong, and the requirement is doing its job of keeping a Superfund site away from the rest of us.

The v0 is just this requirement. If you find that people are lying about it (which they probably will for the same reasons that they lie about Show HNs not being vibecoded or front page slop not being slop eg. https://news.ycombinator.com/item?id=47205208 ), then you can start enforcing it simply by requiring someone else to have run the prompts and found the final result to be reasonably useful and as advertised.

One of the downsides of using an expert LLM to write for you is that they know all that perfectly well, even if you don't, and aren't too bothered by such a chunk. It's like reading any Wikipedia article on mathematics... This is the kind of thing that people are documenting in the LLM-user literature in creating an illusion of expertise (or 'illusion of transparency'). Because the LLM explains it so fluently, you feel like you understand, even though you don't. Hence new phrases like 'cognitive debt' to try to deal with it.

(This is also why people like cramming or lectures rather than quizzing or spaced repetition, because they produce a certain 'illusion of depth' https://gwern.net/doc/psychology/cognitive-bias/illusion-of-... ).

Dude, you literally start the article with

> Andrej Karpathy wrote a 200-line Python script that trains and runs a GPT from scratch, with no libraries or dependencies, just pure Python.

Almost immediately afterwards, you have a section titled "Numbers, not letters". Need I go on?

Interestingly, despite all the AI tics, the opening passes Pangram as 100% human... though all the following sections I randomly checked also come back as 100% AI. So the simplest explanation would be that you are operating adversarially and you tweaked the opening to target Pangram (perhaps through a anti-AI-detection service, which now exist and are being used by the cutting edge, as Pangram is known to be relatively easy to beat, similar to how people started search-and-replacing em-dashes when that got a little too well known), which unfortunately means I now expect you to lie to me in your response since you apparently went that far to start building up clout.

(BTW, how did you accidentally pick 4 rare names which were in the dataset? "Thanks, will fix" is not a real response to that observation. Are you also going to remove all of the 'just pure X' and 'Y, not X' constructions from your posts now that I've pointed it out?)

The review queue is not going to work, unless you have millions of dollars to spare. No one is interested in providing such extremely laborious skilled labor to be thrown away on one-off tweaks of someone's self-promotion of their AI slop, forgotten within the day, and if they are, the ever-escalating flood will soon burn out their remaining altruism.

My suggestion would be more radical: if it's AI written, the code is increasingly irrelevant. What is relevant is the prompt or spec: https://news.ycombinator.com/item?id=47198562 'Show HN' should ban "code-only" submissions if not majority human-written, and allow only "prompt/doc/test only" 'Show HNs'.

I am interested in a vibecoded project which was a well thought out goal or an interesting methodology or a striking prompt engineering trick (eg. the Anthropic Rust C compiler targeting the Linux kernel); I'm not interested in a obvious idea where I'm supposed to be impressed by a huge pile of rotting-code-debt of unknown quality and likely a lot of bugs, which is going to be written by a LLM in a few months anyway.

> "Show me your flowchart and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowchart; it'll be obvious." ---Fred Brooks, _The Mythical Man Month_ (1975)

Naturally, when hobbyists spend a lot of time in a single block to get results (as they are unable to parallelize or meaningfully coordinate over multiple invocations of themselves, due to lacking key cognitive capabilities such as embeddings), they refer to it as 'going ape'.

Oh yes, this is blatantly AI-written. Whether this is ironic and discredits the claims is up to you.

But the opening at least makes a good point: AI-written code is right now something of an embezzlement or theft or fraud. Like junk food, it presents all the appearance of coming with correlates like "an engineer now understands the problem-space and has built tacit knowledge and a theory of how the domain/code work"... But it doesn't. The human engineer takes all the credit (and promotions), having put in a tenth or less the usual work, but evades the blame or tech debt.

This is why vibecoding is the inevitable endpoint. The current code market is a lemon market, and we see the exact same kind of dynamics of 'buyers' withdrawing from 'purchases' as they see all these shiny code bases which are secretly lemons, and sellers increasingly get frustrated they can't get a 'fair price' (like anyone to look at their 'Show HN' posts).

We must understand that the source code is no more valuable than a compiled binary, and that the responsible human doesn't understand it and has to trust it, in the same way that we now understand source code but don't understand and have to trust the compiled binary. Today, I understand my Haskell code, but I have no idea what is going on in the compiled binary (nor do I claim to); tomorrow, I will understand my prompt/spec, but I will have no idea what is going on in the 'source code' (nor will anyone expect me to or admire me for 'writing all that code' or be surprised if some seriously mistaken assumption is hardwired into the generated code). Just as no one cares about a binary you compiled today, no one will care about your source code. (Unless there is something special about it, of course. 'I compiled Firefox for my Linux' - OK, congrats? 'I compiled Firefox for my toaster' - OK now I'm interested.)

So, no one will care about the codebase for your 'Show HN' project; you have to instead get them to care about your proposal or your testsuite. If it's any good, they can simply pass it to their own LLMs to 'compile'; no need to trust your crummy dubious already-obsolete 'source code'...

That was probably a mistake. The religion is great (and I say that as a staunch materialist atheist) and _Fall of Hyperion_ had a lot more TechnoCore and filling out the background than _Hyperion_.

When you are that spectacularly wrong, it is. If you can't update correctly, you probably shouldn't.

Specifically, Cochrane wrote:

> On reflection I have started to worry again. In 10 to 20 years nobody will read anything any more, they just will read LLM digests. So, the single most important task of a writer starting right now is to get your efforts wired in to the LLMs. Nothing you write will matter if it is not quickly adopted to the training dataset. As the art of pushing your results to the top of the google search was the 1990s game, getting your ideas into the LLMs is today’s. Refine is no different. It’s so good, everyone will use it. So whether refine and its cousins take a FTPL or new Keynesian view in evaluating papers is now all determining for where the consensus of the profession goes.

For more recent comments, see https://dwarkesh.com/p/gwern-branwen https://gwern.net/llm-writing https://www.lesswrong.com/posts/34J5qzxjyWr3Tu47L/is-buildin... https://gwern.net/blog/2025/ai-cannibalism https://gwern.net/blog/2025/good-ai-samples https://gwern.net/style-guide

The scaling will continue until morale improves. I advise people to skate to where the puck will be, and to ask themselves: "if I knew for a fact that LLMs could do something I am doing in 1-2 years, would I still want to do it? If not, what should I be doing now instead?"

There is an interesting AI point here: the US Copyright Office recently tried to argue that images generated by a model could not be copyrighted, no matter how detailed the prompt nor how curated, because the artist did not envision the exact output and thus it is merely the output of an uncreative mechanical process. Clearly OP does not envision the exact way in which glass will shatter or frost or crack, and has to repeatedly update and revise based on what happens; are his glass portraits mere mechanical outputs and uncopyrightable?

Oh, if you meant what is the progress indicator itself, see https://gwern.net/design-graveyard#ordinal-word-counts

Attacks can be chained, and this can all be automated. For example, imagine pigbutchering scams... except it's there, similar to some voice-cloning scams, just to get enough data to stylometrically fingerprint you for future reference. You make sure to never comment too much or spicily under your real name, but someone slides into your DMs with a thoughtful, informative, high-quality comment, and you politely strike up an interesting conversation which goes well and you think nothing of it and have forgotten it a week later - and 5 years later you're in jail or fired or have been doxed or been framed. 'Direct methods' can't deliver that kind of capability post hoc, even for actors who do have access to those methods (which is a vanishing percentage of all actors). No one has cheap enough intelligence and skilled labor to do this right now. But they will.

You should write all that up! (I've been around for a while and I never used Flickr except as a casual, but I did know about a lot of mashups and even spent time using FlickrLickr for Wikipedia; yet I've never heard of... any of that.)

The SMPY page is essentially an annotated bibliography: fulltext + abstract + commentary. Since the papers are of highly varied quality, it doesn't much make sense to try to put any particular confidence on it; I am convinced of some things, but definitely not others - particularly the early papers when they had little data and were still experimenting a lot. If I had an essay making a specific claim about SMPY then sure, but I don't really.

The fact that 'certainty' ratings don't make sense for pages like that is part of why these days, I wouldn't have a page like that at all. An annotated bibliography is not an 'essay' and shouldn't be shoehorned into my framework meant for that kind of opinionated writing. I realized that if I was going to 'annotate' a paper, I would either have to go without, or copy-paste it all around indefinitely and it'd violate DRY and be a nightmare. Long story short, https://gwern.net/doc/iq/high/smpy/index is closer to what that page should be, but it's a lot of work to sit down and convert the legacy page over to pure annotations, so, it is what it is. Maybe a LLM can do it for me soon - it seems within the ability of Claude Code.

> I'd note that there are many studies showing that the usual outcomes for gifted kids are not all that great.

No, there's not, and they do do great. And this goes back to Terman: there's a handful of highly selected examples (eg the Australian kids recruited from child psychologist referrals, the self-selected self-diagnosed Mensa adult survey), furious anecdotes, and then every systematic prospective sample like Terman or population registers or SMPY shows the opposite.

And while Tao is, of course, exceptional, the results for accelerated gifted kids are generally great. And Tao was part of SMPY (note the URL path, supporting documentation for https://gwern.net/smpy ) and helped demonstrate this in practice.

I enjoyed the reward-hacking one: https://pagedout.institute/download/PagedOut_008.pdf#page=59

K-V caches are large, but hidden states aren't necessarily that large. And if you can run a model once ridiculously fast, then you can loop it repeatedly and still be fast. So I wonder about the 'modern RNNs' like RWKV here...

> Ideally, I'd like to continue working on it to build something that can help clubs make minimal, inexpensive changes while maximally improving the strategic interest if the way the course plays.

Yes, that's why I mentioned that this gives you at least one possible measure of 'strategicness' which can be computed automatically. Measure the variance across policies (skill level, which maybe you could define physically as some sort of 'mean absolute circular error' in strokes?), and now you can do quite simple optimization routines to modify courses. Like take a course, randomly flip some squares to sand/water/grass/etc, compute the new strategicness, and keep it if it's higher. Random search, simulated annealing, CMA-ES, novelty search, lots of easy possibilities I bet even a LLM could implement for you these days which would let you take an existing golf course and search for new modified layouts with higher strategy, to inspire a human expert in modifying a course.

I'm not sure this is even measuring LLMs in the first place! They say the definition is "big data analytics and AI".

Is putting Google Analytics onto your website and pulling a report 'big data analytics'...?

OK, so assuming you clean that up a bit and this becomes officially supported in SingleFile/SingleFileZ, what is missing compared to Gwtar? Anything important or just optional features like image recompression and PAR2?

Allowing more metadata might be useful. You can add anything to the manifest at build time as assets are not required to be loaded or ever used (because this is impossible to statically check). I suppose we'd have to define an official prefix like 'gwtar-metadata-*' with like a 'gwtar-metadata-screenshot' and 'gwtar-metadata-desciption'... Not obvious what the best way forward is there, you don't want to add a whole bunch of ad hoc metadata fields, everyone will have a different one they want. Exif...?

Multiple versions or multiple pages (maybe they can be the same thing?) would be nice but also unclear how to make that. An iframe wrapper?

I considered and rejected deduplication and compression. Those can be done by the filesystem/server transparent to the format. (If there's an image file duplicated across multiple pages, then it should be trivial for any filesystem or server to detect or compress those away.)

If those were guaranteed inaccessible, wouldn't a web browser be within its rights to optimize those away?

My immediate thought is that OP is reinventing dynamic programming/RL from first principles. The final visualization looks exactly like a standard value estimate heatmap. Golf is a MDP over all the physical points on the course, with stochastic probabilities of transition to each one based on golfer skill and physical randomness. Strokes are the cost to be minimized, the colors are the value estimate at each state, and his difficulties with the different maps is because a value function is defined as the expected value of being in that state assuming you will follow a particular policy thereafter (ie. be a golfer of a particular skill level, playing optimally for that skill). This lets you formalize 'strategicness' of a golf course: it is how much the value estimates differ on average across the full range of golf skills; a non-strategic course looks identical for the beginner and pro, while an incredibly strategic course might have completely different values for every point for every bracket of skill. (You could probably automate creation of pathological golf courses this way, where even a slight increase in skill makes the new strategy different.)

But that's a reason you should expect it to stop working soon, just like all the older tricks like "my grandmother will die". If you have a universal 'blind' prompt which can increase performance a little bit... the AI labs can just toss that into the training loop to teach the model to do it automatically, whatever 'it' was, like 'trying harder' or 'writing down a useful idea'. And then the prompt stops working because the next generations do it by default.

(This also suggests that you should expect them to generally be bad at judging novel self-generated prompts/skills - if they could judge those, they would already be using them! There is a generator-verifier gap, but it is already exploited heavily during post-training and not much low-hanging fruit left there.)

Yep: "The tower itself isn’t just a gimmick; it’s a physical presence that creates tension and anticipation the moment it hits the table. Add the Dark Horde expansion and suddenly the game isn’t just something you play—it’s something you engage with." 100% in Pangram, BTW.

But there's enough biographical details and concrete facts that I think he at least wrote the outline before having ChatGPT rewrite it.

I think there's still something missing here. This is a strange place for ChatGPT to confabulate quotes: extracting short quotes from a short text blog post is about easy as it gets these days. GPT-5.2 Pro can handle tens of thousands of words for me before I start to notice any small omissions or confabulations, and this was confabulating all that at just 1.5k words?

So since he says he was sick and his recollection cannot be trusted (I don't blame him, the second-to-last time I had COVID-19, I can barely remember anything about the worst day - which was Christmas Day), something seems to be missing. He may not have pasted in the blog post like he remembers. Or perhaps he got routed to a cheap model; it wouldn't surprise me if he was using a free tier, that accounts for a lot of these stories where GPT-5 underperforms and would explain a lot of stupidity by the GPT. Or didn't use GPT at all, who knows.

Yes, network is assumed. If it's local, there's no problem, just use MHT or SingleFile!

> Maybe this is downstream of Gwern choosing to use MediaWiki for his website?

This has nothing at all to do with the choice of server. The benefit of being a single-file, with zero configuration or special software required by anyone who ever hosts or rehosts a Gwtar in the future, would be true regardless of what wiki software I run.

(As it happens, Gwern.net has never used MediaWiki, or any standard dynamic CMS. It started as Gitit, and is now a very customized Hakyll static site with a lot of nginx options. I am surprised you thought that because Gwern.net looks nothing like any MediaWiki installation I have seen.)

I see. You could probably build something on top of wabac.js... But you'd need some sort of multi-file setup to support the indirection, I suppose.

> I imagine that loading archives without a web server was probably out-of-scope for Gwtar

More that it's just not important to us. I don't even look at the archives 'locally'. They are all archives of public web pages, which I just rehost publicly. When I want to look at them, I open them on Gwern.net like anyone else!

And if I really needed to, for some reason, it's literally a Bash one-liner (already provided inside the Gwtar as well as my writeup) to turn them back into a normal multi-file HTML. (This is a lot more than you can say for a WARC...) So my reaction to the complaints about lacking local viewing is mostly just ¯\_(ツ)_/¯

> (e.g., putting the tar in a <textarea>'s RCDATA; I wonder how well browsers support "binary" data in there?)

I don't know the details but you can just base-encode them, so I suppose that's an option, as long as you rewrote the ranges appropriately, maybe?

(Also worth noting that you can go the other way: if you really desperately want to preserve the raw header responses, you can just use the flexibility of Gwtar to append the WARC to the end of the file. As long as the range requests work, users won't download that part. The duplication is not so great for long-term storage, but you can just XZ them and that should remove duplication and overhead.)