jerf

Read more papers.

I've also read some summaries of even more papers, but yes, actually read papers, beyond the abstract.

Let any who read this be aware: You should be immediately suspicious of AI code that is much much faster than established code bases. AIs are even better at benchmarkmaxxing than humans are, and humans were already pretty good.

It is by no means impossible for an AI-written code base to be faster than a conventionally-written one, of course. It's just a time to be suspicious. Especially if there's little to no evidence that the human has subjected the code base to any further testing of their own. AIs seen to be able to benchmarkmax even in the presence of a lot of unit tests that would generally speaking have prevented a human from going too far down that road.

I have read more paper in the last year than the last ten years. Across a wide variety of subjects. While the AIs still need open access to science in order to function, they have make that open access something I can actually use like no other technology before it. Nothing has opened the scientific commons to me like AI.

There may be a concern going forward with locking things down. One can lament that AIs need to be prompted to "hit the literature" directly or they don't really think about it, prefering random web pages most of the time. But at least at the moment science is more open and accessible than ever before.

Lottery tickets at least avoid the worst aspects of the real time feedback that things like slot machines can give. Though scratchoffs certainly get closer.

I do hold out some hope that in this pendulum swing we can land at least for a while on banning state sponsored gambling, though. It'll come back. Too appealing to keep down forever, unfortunately. I like to say it's a good exercise to sit down and really think about the difference between "organized crime" and a "government" but when a "government" does things like zealous defense of its gambling monopoly it does rather weaken the distinction....

It isn't that it will have been trained on the source code. The "decompilation" craze has it in the name... it is being done by decompiling the game and then modifying it. I don't think it's even a question as to whether or not the game companies are going to come down hard on this and I would not recommend to anyone that they jump up and volunteer to be the legal test case for that.

A clean-room implementation is going to be much, much more effort than a "decompilation". Even a decompilation is, I think, more than the hype is claiming. You can't just point at a game, say "decompile it and make it work on Windows" and get a result in one shot. It will take playtesting, cycling around, QA, and time to polish it up. But that will still itself be much easier than an actual clean room implementation. Let alone getting sued by the owner anyhow and having to prove it was clean room in a typical chaotic individual dev environment.

There are a number of things that cycle because the solution ends up being successful enough that the next generation doesn't realize there is a problem being solved. Chesterton's Fence, except on a generational timescale it's Chesterton's Oscillating Fence.

100% squashing gambling has generally not been successful, but the strategy of fencing it away from most of society and keeping it in a set of defined places that require people to actively seek it out has been successful enough that people forget why we don't allow gambling in every convenience store in the country. I'd say the additional "convenience" of having it in your pocket all the time is a new nuance, but doesn't necessarily change the fundamental equations.

We will relearn. Our grandchildren will forget again. (Unless things have really changed.)

"despite that this isn't some universal constant in the training data"

Isn't it though? It's not exactly an uncommon answer to the question. Basically the same input given to basically the same algorithms yield basically the same result isn't really that stunning. They certainly aren't independently reaching this answer, their paths to this answer are extremely, extremely heavily correlated.

If they gave a really different answer that wasn't heavily represented in their training data that would be at least something interesting. But this is basically what I'd expect.

It doesn't mean they're right or wrong. Humans think it about other humans for good reasons too. It just means it's not that surprising, which to put it another way, doesn't carry much information.

"You can't fire me, I quit!"

In Chesterton's fence analogy, the person who wants to tear a fence down is at least aware there is a fence there. I think a lot of modern UI is being done by people who aren't even aware that there was logic being used to make decisions and are just steamrolling over the fences without even being aware they existed.

In their defense, it's not like good UI has ever dominated very many spaces. It is very hard to come up with a concept of good UI concepts from a natural meandering through the shattered landscape of UI paradigms anyone computer literate ends up subjected to. And then it's even harder to convince a manager that it's worth so much as a moment's thought about such things. Not impossible. But not easy.

Exclusive to Meta (unfortunately), there is Triangle Strategy: https://www.meta.com/experiences/triangle-strategy/534013047...

Control is a bit dodgy, I kind of feel like it didn't quite commit to either the VR or the controller-like controls, but it's perfectly playable and is an interesting angle on things.

Is there anywhere good to get some sort of reliable stream of information about this possibility?

It's going to be pretty bad if we all panic to get the latest PHP up, and all confidently roll out our updated websites, just for them to still be vulnerable to something serious anyhow.

It creates an absolutely enormous political snarl to decide who is "worthy" of getting advance notice on something like this, and to correctly determine who is capable of keeping the secret in the presence of highly motivated active attackers who may have capabilities all the way up to and including "state level actors".

Basically, it's only possible in our imaginations where we don't have to get into the nitty-gritty of exactly who gets what notifications. It isn't possible in reality for something this large and consequential.

I don't expect every person to instantly fall in love. I expect that it could be a more popular option than it is now if the industry hadn't been quite as monomaniacal about VR being "VR".

It doesn't help that the headset hardware space has been closed off even worse than cell phones. One of the things I hope the Steam Frame will do is crack open the space of possibilities more by virtue of being a headset that is open, and force even the closed headsets to expand their thinking. More possibilities won't hurt you, even if you are personally uninterested in them.

Since getting my Frame I think I've spent more time playing Super Mario Galaxy 2 in 3D than I have doing anything else. 3D in a window is much easier to deal with then full VR from a motion sickness perspective.

I think the VR space made an unforced error by focusing a bit too much on VR and not enough on just plain 3D. VR is cool and it has a lot of interesting applications. But I've enjoyed my time with SMG2 in a 3D window quite a bit. I've enjoyed Triangle Strategy on my Quest 3S, which is a tactics game played on a diorama. There are some other applications based on the idea like shared dungeon crawlers modeled on table-top layouts rather than first person perspective. There's a lot of games that need only minimal tweaking or fiddling that could add a bit more visual interest by being displayed in full, true 3D. The space of games that could be improved by just being playable in 3D is much larger than the space of games that can be improved by VR specifically, since the former is a superset of the latter.

Subconsciously it may feel like if you're developing for a "VR" headset, it ought to be "VR", but that's not the case. It's a well-known trap in gaming in general that when some new tech comes out there's a rush of things that feel obligated to "exploit" the tech so as not to be perceived as being behind the times, but it always becomes clear that is not necessarily. Most famously, the numerous Wii games that had to integrate "motion controls" and turned a nice, easy, concrete button press into a vague, nondeterministic "waggle" until just saying "motion controls" with a certain inflection has become a meme. There's no problem using the VR headset to enhance a game... or indeed any application... without it having to be "VR".

My Steam Frame was cheaper than this 57" curved "gaming" monitor: https://www.amazon.com/dp/B0CDQM55C9 If gamers are willing to drop that on nice monitors just to play games, it would make sense to drop that much on a VR headset to play them in 3D too... if there was anything to do with that.

It's an interesting study idea. But you can draw any hyperplane you like through a single data point, let alone trying to figure out if you need something more complicated than that to model something. "Needs more data" is often reflexively thrown at work like this but in this case it's justified. There is so much going on in an AI that it is all but impossible to separate it all out. Do western open weight models do this in the other direction? It's entirely possible. Perhaps even likely. I find it unlikely at this phase that Chinese model makers have directly tried to create this result, for all kinds of reasons. Speed is paramount in the AI space right now. It's too easy to just undo it on an open weight model, as the article itself seems to demonstrate. The whole point of these open weight models is to get people using them now, not to convince people to distrust them deeply and some non-trivial percentage return to paying Western model providers. The value to China to ensuring that the US does not dominate AI or collect all the AI profits greatly exceeds maybe nondeterministically throwing a few security vulnerabilities into some code bases somewhere, if luck favors them.

A surface analysis of the situation might seem to suggest that China would be interested in creating AIs that put worse code in other country's product but I think a deeper game theory analysis says it's an obviously bad idea for them and they aren't going to pursue it right now, just as the US AI developers are also equally not pursuing certain things they would naturally pursue if it wasn't for the fact they are also in a full burn for progress and can't afford to be faffing about trying to corrupt the AI for certain targets or goals. Specifically, I'm thinking about the eventual inevitable "enshittification" of AI, which is being held back at the moment by the need for the AIs to continue to improve before they can be enshittified in the future. It's the same basic analysis on that side. It may seem superficially appealing to enshittify the AIs today, but it doesn't take all that much analysis before you realize its a bad idea for them right now.

"I have not come across a single piece of software that has improved in quality after the developers start using AI."

That trivially maps on to the exact argument being made in that video. You think you haven't come across any software that has improved since the developers starting using AI. But if you encounter improved software, you have no mechanism to determine whether or not it was improved by AI. So you decide without even consciously thinking about it that it isn't because of AI. So of course you have not encountered the thing your sensors can not sense.

And just like CGI, there are some characteristic failures that allow you to realize that the failure is due to AI misuse. So you do identify AI failures, and as a result, all the AI you can identify is a failure.

But your sensors could never have given any other result. So it's not a surprise that your sensors give you that result. The result has no meaning.

Here's an 11 year old video trying to educate people who think they "hate CGI" just how much CGI they've been watching, just without realizing it, because it was actually quite good: https://www.youtube.com/watch?v=bL6hp8BKB24

We are almost certainly already there with AI coding, and if we aren't I give it six months to a year tops. AI is being used for coding everywhere now. The difference between the slop and non-slop isn't whether or not AI is being used. It's how much care is being taken by the developer. But it's no longer a question of whether new code had AI contributions. It did.

By the simple mechanics of information theory, there's always going to be certain common grain to what you get if you try to prompt entire massive systems into existence with minimal human input as to what the system should be. You'll always be able to type in "give me a 2D platformer like Mario but with cats and barn animals instead" and get some sort of slop result, no matter how good that slop result may eventually become in the abstract. It'll take more information from somewhere to make it something other than that slop result.

(Though that information may not all source from a human sitting there and batting things back and forth with an AI, though that will be an effective method.)

I'm confining myself to "code" here, which is mostly useful to a game because of the results of the code. Gamers do not sit there and actually admire the cleverness or cleanliness of the code. I'm not speaking to art, which is a more complicated scenario. You can make a similar case for visual and audio art, but I also don't think the AI stuff is as far along relatively speaking as it is for code.

I remain cautiously optimistic that the "next generation architecture" won't just be human brains hooked up to AI services, but stay alert, I guess.

Yes, and when you ship a Go WASM package it is already included. But it has a certain impedence mismatch with WASM since it is rolling its own stuff that WASM can't see, I believe the performance is a bit less good than it could be, and it results in very large WASM packages for small amounts of code.

Since a typescript compiler isn't "a small amount of code", you'd have to see specific numbers on the final WASM assembly size to know how relevant that is. I'd expect well-written Rust code to beat well-written Go code on final assembly size by some amount, which may or may not be relevant to any particular use case (since in a lot of cases one "couple dozen megabytes" is much the same as another when it really gets down to it, it's not like Go is going to be emitting gigabytes to Rust's hundreds of kilobytes or something). However I can't judge what AI-written Rust code versus well-written Go code would come out as. AI is not terribly good at being concise (he said, while writing this post between prompts fighting with an AI to make a PR much more concise than it currently is).

"Obviously this will never happen."

It will never happen with LLMs, but we can hold out hope that it will happen someday. Humans do not become what we are by having the whole of the internet poured through our heads multiple times over. It seems almost necessary that there is some next-generation architecture that won't require as much training data.

Probably still enough that it'll stagger a home connection trying to download a torrent or something, but perhaps not so much that it is insane to also release the core data set one way or another.

In 2026, everything is a stream of numbers. It's obvious everything is a stream of numbers. We can save them to disk and stream them over the internet and archive them and it it were't for DRM and hostile practices we could stick them all in the same place right next to each other in the same directory.

In let's say 1996, we weren't remotely there yet. A radio program was distinct from a television program was distinct from a book was distinct from a CD was distinct from a CD-ROM was distinct from a video game... and so on and so forth. You had a tool for recording from the TV, the VCR, but that didn't do the radio. You had a tool for that, which was a cassette tape, but that didn't do anything else. Except for certain early computers who took advantage of the fact that a cassette tape is really just a recording of varying voltages which can be fed into a computer to be interpreted as numbers, but this is one of the rare exceptions where there was some crossover, and by the 1990s this was gone anyhow. Now you had floppy disks, which were for computers, and they didn't store movies or books or radio shows, they just stored computer programs.

Everything was compartmentalized so the idea of "storing a computer backup on a VHS tape" sounded like some sort of forbidden union.

What sort of fascinates me about this history is how so much of it is written into the law now. We have all these different laws where music is treated this way, but movies are treated that way, and books are treated some other way, and so on and so forth. Copyright law isn't quite entirely broken up into unrelated fiefdoms but neither is it anywhere near as unified as it would be if it was a brand new idea someone was coming up with today. And it still has trouble dealing with the concept of things like "databases" sometimes.

Now it's not such a weird idea that a VCR tape is really just a stream of numbers to be sliced and diced as needed for a project like this. But back then, so much of the processing involved fundementally analog processes that anything that crossed the streams seemed like magic. I don't feel like I've conveyed the idea properly that this wasn't just a technical thing, I'm talking about how people thought about these things. It was a revelation at the time that CDs could contain "video". It was a revelation that you could have "an MP3 player" that had "computer files", but, but, it played music. You could store backups on a CD. You could put a book... on a computer. I remember the first time I realized that if I took these cables from the VCR and fed them in to the input of my cassette player I could record the audio off a VCR on to a tape, the point being not that this was impossible but that it was something I had to realize. We had all these categories in our minds with strict boundaries between them that don't exist anymore, because those categories are just accidents of history.

What we have today is much closer to reality... it's all just numbers. Any numbers can go on to number storage devices. Any numbers can be streamed across the number delivery service. The tech industry is in fact busily trying to re-establish the boundaries they themselves erased so they can artificially segment the various markets again.

It's fun to see an old-school boundary breaker project again.

One of the things I've started noticing at work is that for the last few months I was banging away at having multiple streams at a time. But over time, I've reached the point where there is some human blocker on the stream, be it QA to test it, other humans to have attention to pick it up for the next step, deployment of some sort, or in some happy cases, it made it through to deployment and the problem is now solved. In the past couple of weeks my multistreaming is toning down and more often I'm back to one stream of work. Often a much higher level stream than I would have been working at a couple of years ago, and I'm moving faster in a number of ways. Some of that "faster" has been burned up in things like changing environments that I previously just wouldn't have done, though.

Open source has had a QA problem pretty much its entire life... not in the sense that it stinks necessarily, but in that getting it at all is a challenge and was in many projects probably already the blocker to further progress.

One wonders if what is happening here is just that even in the software space, where the fluidity and flexibility is maximized as we work with nearly entirely digital constructs, that once we burn through the dry tinder of "all the stuff we know we needed but didn't have the coding time to get to" and the frontier of "work to do" advance on all fronts to "some sort of human or real-world blocker", it may just turn out that the steady state improvement in productivity after accounting for Amdahl's Law, even when code is "nearly free", is only a maybe 2x or 3x improvement after all. If this is true, goodness help the financial bubble which is assuming that something like 5x or 10x indefinitely is a lower bound, if even in the best possible domain for AI that's all we really get.

You may be interested in https://github.com/ergo-services/ergo . Which goes so far as to be interoperable with Erlang directly.

First, I've already thought about the second-order effects and disagree that the second order effects are that all software vendors in all spaces get wiped out: https://jerf.org/iri/post/2026/what_value_code_in_ai_era/

Second, if "all software vendors in all spaces get wiped out" was going to happen I think we'd be seeing more of it already. Yes, there is support for some software vendors possibly getting wiped out... but, then again, you can't look at a bad vibe coded implementation of "the entire Adobe suite" that was rather overblown in the press coverage and forget that Adobe also has AIs. Instead what I'm seeing at my own workplace as we ponder the question of "should we replace this service we buy with some local vibe coding" is still remaining but we don't want to support that. Can we replace Confluence with vibe coding? Yes, but that was true five years ago too and we didn't want to do it then, either. We still don't want to do it. HN commenters have always overestimated the desire everyone in the world has to become programmers and take responsibility for everything.

One wonders what percentage of those who are certain that vibe coding is going to nuke all software vendors can't even be arsed to self-host a Jellyfin or VaultWarden install... which is way easier than vibe coding the whole of those things from scratch, and we're talking about people who are already programmers. Physician, heal thyself.

At this point I'm ready to call it... LLMs aren't going to ruin my job. They're amazing, they're useful... and the problem remains that to a first approximation human beings are uninterested in spending all day talking to a computer, whether through a classic IDE or through an LLM. Specifying a problem enough for an LLM to solve it, even if we stipulate one that does it perfectly, is already more than most people want to do.

You also aren't going to get an accountant running 4 simultaneous streams of AI-driven development and working through an AWS issue when something goes wrong.

No comment as to the next generation of AIs.

Are you a multi-trillion dollar company that the government considers a vital national security interest for the forseeable future?

No?

Then yes, you can be held responsible and it's not even a subtle question in the law. There's no one else around who is going to be held responsible. Certainly not the aforementioned vital national security interest.

"Vibe coded" and "supposed a lot faster than the competition" is becoming a smell to me. LLMs have a demonstrated capability to benchmarkmaxx that is even greater than human's already substantial capabilities to do so.

I'm not saying this isn't possible, I'm saying I'd wait a bit before I got too excited. That it seems to have held up to some other commenter's testing which was presumably not in the benchmark is an interesting sign, as a lot of time benchmarkmaxxed code falls over dead the instant you give it anything not in the benchmark. But I've still seen a whole bunch of projects "X times faster than what we told the AI to copy" that were less than met the eye to jump immediately to this must be awesome without more people trying it out seriously.

Is the Typescript not garbage collected somehow?

If "scriptc" is so invasive it makes Typescript no longer even a garbage collected language then it really isn't Typescript anymore.

The problem will remain due to information theoretic concerns. It is not possible for an AI to read minds. You can say to an AI "build me an accounting system for a small tax firm" and we can, for the sake of argument, even stipulate that it will produce a functional system based strictly off that prompt. However, no matter how smart the AI, the result can not be any more detailed than that. Did you want it for MacOS? It may have built it for windows. Did you want it integrated into an already-existing database server? It can't have done that, and since you didn't specify the database you want it could have guessed wrong, and certainly can't guess about what tables you may already have you want to integrate into. It probably didn't guess your jurisdiction correctly (unless it was fed that in some other contextual prompt). It can't guess your most common work flows and optimize them. It can't pick out specialized features you want just for your special use case.

None of that is because AIs aren't good. It's just plain information.

An AI could sit there and interrogate the user for this information. In that alone, it would already have exceeded what a lot of people are going to want from a system like this. People, normal people, want AIs to magic stuff into existence, not spend hours interrogating them for a design requirements elicitation. That's a barrier a lot of people will not cross, and the AI companies, who fully realize that, aren't building that way.

Normal people do not want to even do the work it takes to fully specify a problem, even if an AI walks them right through it, let alone the verification that the work is correct, and actually what they intended rather than what they asked for.

And of course, in practice, AIs are not emitting bug-free code, miss entire requirements at a shot, and have a variety of other characteristic failure modes in this space that I expect are fundamental to the LLM architecture and aren't ever going to leave. If they continue to be able to make bigger systems they will simply continue to make correspondingly big errors and omissions in the process. No comment as to what the next generation AI architecture will do, though.

Probably the biggest reason "hackers" got civilized is just a regression to the mean effect. When you have something extremely difficult to get into, there is going to be some strong selection effect that is going to accidentally select for some other correlated traits. As the barriers to entry dropped, more people get in, so the selection gate drops as well and you get something closer to the average human. You can see this in all sorts of niches over time. For instance, while little music niches will often blame their unique culture getting diluted on "selling out" I think it is often the case just that they attracted more people and got regressed to the mean.