jiggawatts

Somewhat over-estimating the numbers, but even if your stats were accurate it wouldn’t change the critique that “the west” does the exact same thing.

This isn’t what-about-ism, this is simply acknowledging the realpolitik that there are great powers vying for control over the rest if the world through proxies. A game as old as civilisation itself.

We support Ukraine, they support Russia.

We support Israel, they support Iran.

We X, they Y.

Neither is more right or wrong than the other, and neither is the only bad actor.

The same FIFA that made a documentary about how their leadership were battling corruption… a year before they were arrested on corruption charges? https://en.wikipedia.org/wiki/United_Passions

Show them this video and they’ll immediately commence a mass sterilisation program, like we do with invasive mosquitoes.

Literally:

    BEGIN TRAN
    BEGIN TRAN
    COMMIT TRAN
    ROLLBACK TRAN

"Latency" is my answer to the Fermi Paradox.

As species become more intelligent, develop digital technologies, and their equivalent of the Internet, they become interconnected.

Interconnectivity is limited by the speed of light. Even here in Australia, the "lag" is very palpable. Conversely, whenever I visit the US, suddenly every web site provide instant satisfaction instead of frustrating delays.

As the speed of artificial intelligence increases, these delays become exponentially limiting. There's a point where you want to be in the same city, or even the same data centre as everything and everybody else. (US-East-1 comes to mind.)

Why would anyone voluntarily isolate themselves from the interconnected web, separated by positively glacial light-minutes, hours, or... gasp... centuries!

If the speed of light is an insurmountable barrier, then nobody in their right mind[1] would cut themselves off like this, from everything and everyone else.

Even huge "generational ships" would be like... a small rural country town compared to a glittering, vibrant city. No matter how much static information you crammed into its computers, no matter how much entertainment you brought with you, it would be less than that available on the home planet.

It's simple incentives: Why would anyone choose solipsism and isolation, at huge cost and risk, for effectively zero upside!?

Space is not the wild west. There are no prairies, no fertile lands, no rivers to fish. Certainly not in the relative sense compared to the towering cultural edifice that a multi-million-year-old civilisation could build up on their home planet.

Imagine being an artist, or a performer working at some cultural centre of the world like Broadway moving into the middle of totally empty desert.

"Woo! Isolation! No crowds! No fellow performers! No money! Yay! Finally!" -- said nobody ever.

[1] The corollary is that any members of a species willing to head out into the vast cold uncaring abyss of deep space are insane to a significant degree. Space is filled by crazy, dangerous aliens. The sane ones stayed at home!

Database engines I've worked with generally support nested transactions, so you can open a transaction, open another transaction, commit it, then roll back the outer transaction... and it's like nothing ever happened.

This ability to compose transactions is their main benefit over other kinds of concurrency control!

Can you imagine trying to explain any of the current world leaders to little green men?

“After they killed their first hundred thousand other people, why weren’t they arrested and jailed?”

“Err…”

“And why not arrest them after they stole billions from the public they were supposed to serve?”

“Umm…”

“I can understand voters making a mistake once… but repeatedly voting in the same autocrat?”

“Well…”

My mental model is this:

There is a vast ocean of human knowledge, far beyond the capacity of any human brain, even within specialised fields.

Books helped "plug the gaps" in our knowledge, increasing the scope that a single human mind can encompass.

Web search engines did the same thing, but more and faster.

LLMs are like search engines on steroids, essentially a research librarian that operates at 1,000x human speed and can "in context" locate relevant information, adapting it to fit the hole it needs to go into as well.

It feels less like discovering new theorems, but instead having direct access to all theorems, which is hugely valuable in itself.

I.e.: the recent counterexamples to open conjectures has largely been about the AIs "trawling through all the things" and scraping together every bit of human-generated knowledge ever produced that is relevant to the conjecture.

Conversely, in the past, we had to "make do" with sub-standard solutions where the problem had been solved, but finding every relevant solution in the ocean of knowledge was prohibitively time consuming.

In some sense, LLMs will "raise the floor" in what is considered the minimum level of quality of a solution, where even throwaway / toy designs will now start applying every bit of accumulated wisdom instead of just some of it.

We have mechanised attention.

Read it twice already!

Something I noted -- trying to avoid spoilers here -- is that Rorschach is awfully similar to LLMs, but the book pre-dates ChatGPT by several years, which I feel says everything you need to know about how well the author foresaw the future.

It is well established that evolution is driven not only by fitness in the trivial sense of "survival or death", but in large part by mate selection.

(Disclaimer: The rest is a personal theory with zero scientific support.)

This begs the question: selection based on what? In many bird species it is the complexity of the male's song, or the colourful beauty of his feathers. There are "dances" and courtship rituals in many animals, including even insects.

Humans almost certainly also subject to evolution caused by the evolutionary pressures of mate selection!

This is best said by Anthony Hokpins in Westworld: https://www.youtube.com/watch?v=OjsGJOXhPGU

"I read a theory once that the human intellect was like peacock feathers. Just an extravagant display intended to attract a mate. All of art, literature, a bit of Mozart, William Shakespeare, Michelangelo, and the Empire State Building—just an elaborate mating ritual.

Maybe it doesn't matter that we have accomplished so much for the basest of reasons. But, of course, the peacock can barely fly. It lives in the dirt, pecking insects out of the muck, consoling itself with its great beauty."

Which is a good bit of drama, but... if true, what are the consequences for the evolution of the human species?

Assume for a moment, that yes, women are attracted to male intelligence, which men show off with music, art, and "great works".

The obvious immediate effect is that this causes the average intelligence of Homo Sapiens to increase over time.

But the secondary effect is more interesting: judging intelligence benefits from more intelligence! As the next generation of women inherit greater intelligence, they become better judges of male intelligence, more picky and precise. This forces male intelligence to increase ever faster to keep up with the "rising demand". Rinse and repeat.

I can't prove, but strongly suspect that this is the case.

Birds of Paradise become "more beautiful" on the scale of aeons because more beauty in the males doesn't improve the judgement of the females.

Humans become more intelligent on the scale of mere millenia because more intelligence does improve the judgement of the females.

Look at it from the perspective of a manager above individual developers. Their instructions to those devs are already a series of non-deterministic incantations. They ask for things in English and get something back in Git a week later!

We've just pushed that indirection down a level from managers to ICs.

The ICs are shocked and surprised that this level of imprecision is allowed.

Their managers are not shocked at all, this is normal for them!

I was just thinking of this exact comic, which is etched into my brain for some reason.

The obvious counter to this is that we've been going through this evolution of increasing abstraction as developers for nearly a century now.

In the 40s and well into the 60s, most code was written either as straight up machine code or an assembly language. MS DOS is almost entirely assembly.

UNIX ushered in the era of "high level" portable languages like C, Fortran, and Pascal that some developers hated because they felt like they were losing the fine-grained control that they had with assembly. The compilers just "weren't as good" as humans at optimisation!

Then the compilers got better and people started using garbage-collected languages like Perl, Python, Java, JavaScript, and C#. Similarly, many people bemoaned the lack of control over memory allocation, lower efficiency, etc.

We're simply stepping up to the next level of abstraction.

Look at it this way: decades ago when I first discovered C++ templates, it felt like waving a magic wand in the direction of the computer. It blew my mind that I could simply substitute "float" instead of "double" in between some angle brackets and the compiler would write reams of code for me!

We simply have better magic wands and more powerful spells now.

Dolphins sleep with one hemisphere of the brain at a time.

Sharks never stop swimming because it’s the only way they can move water over their gills.

I knew the situation was bad, but I didn't know it was this bad. What an illuminating comment indeed!

Fibre to the apartment, but it is only “lit up” by a non-NBN provider.

The contract that the building manager signed.

I love being ripped off because I'm renting, so instead of having a direct relationship with my service providers, they are legally allowed to sign binding contracts with the building manager who I'm sure in no shape way or form receives a kick-back.

So I'm stuck with an energy provider that is too incompetent to figure out how to bill me correctly, but puts a markup on what I'd pay as a home owner, and I don't even get the NBN despite having fibre to the premises!

No IPv6, no gigabit Internet, no free solar electricity.

A language controlled primarily by Oracle is certainly... a choice, if you prefer avoiding traps controlled by megalomaniac overlords.

> people who think we should remove pages from knowledge book.

"No knowledge in Kamar-Taj is forbidden. Only certain practices."

You reminded me of that quote from the Doctor Strange movie.

Even the most "dangerous" knowledge can be life-saving, depending on the circumstances. When I was a teenager, I read the Terrorist's Handbook, which was a text file circulating on bulletin boards. After 9/11 it vanished, because posession was considered a crime in some jurisdictions.

Knowing how to build a bomb is useless and/or dangerous in civilian life. Ask a Ukrainian if it is useful knowledge!

Knowing all about nuclear weapons doesn't mean you're going to blow up a city. But if someone else does, then you'll have the knowledge needed to avoid the worst of the radiation and maybe survive.

> It runs and accepts people's payments, which means you're not on the front page of the newspaper (not in a good way).

"It works if nobody attacks it." isn't security.

> They also have to go through payment networks which are very often the limits on those things.

For the same reasons.

> need Visa and Mastercard to upgrade as well.

They won't, and it's not worth asking them to. They're dinosaurs and will simply be replaced by a newer, more agile competitor.

It's already happening! Billions of people in Asia pay with their phones using home-grown payment systems, most of which are generally much more modern and better engineered.

> It is signed with the other party's private key. What does that buy you?

Same as what HTTPS does: attestation of identity by some trusted third-party, to some non-zero level. This could be literally just the existing CA networks and DNS domains as identity, but it could be governments, the banks themselves, etc.

I had some fraudulent transactions on my account labelled "Microsoft Subscription". It wasn't Microsoft. How can I tell?

Not even the bank knew the identity of the third party!

That's insane.

> Because you know that the current one works.

What do you even mean by "works", specifically?

> it might have bugs, you know what they are.

Okay, so it doesn't work, you know it doesn't work, it's just that you accept the specific ways in which it doesn't work.

I've lost track of all the myriad stupid ways in which these ancient systems are hugely ineffectual without even being outright faulty.

Like airline tickets where your name is printed as "LASTFIRSTMR" in all caps and no spaces because their systems are ancient beyond belief.

Similarly, my bank statements are security-critical, because anyone with a copy of my credit card details can pull money out of my account without my express authorization. But...

... because they're stored in terrible ancient mainframe databases, the text fields all have tiny maximum lengths. Hence they're all abbreviations. Attacker-controlled abbreviations without any authenticity assurance of any kind!

I have no idea who actually transfers money out of my accounts! There are no URLs, no metadata, nothing to actually confirm the identity of the other party. Every field in a transaction record is 100% attacker-controlled and unverified by my bank.

If you look at it from the perspective of someone used to modern web security, then you realise that banking is a raging tyre in comparison. Banks literally just accept a certain rate of criminal activity and "price that in", reversing transactions when asked -- which itself can also be a criminal activity. They just shrug their shoulders.

"What can we do about this?" -- says the people that have tried nothing and are all out of ideas.

Rewrite it. The whole thing.

Use an actual database, something made in the last three decades instead of half a century ago.

Use cryptography. No, not crypto coins! I just mean a bog-standard algorithms like public-private key signing so that it is possible to confirm the source of transactions.

Etc.

I would much rather have something generated with the assistance of a modern LLM than what we have now, which is security holes big enough to drive a panamax container ship through.

I don't bother to use open-weight models, but for all of the "security" or "security adjacent" work I have tried recently, GPT 5.5 x-high has been the first model that produced useful output.

Even very old models could spot the most glaring issues, but it's a different story if you scan a source repository where humans can't find security vulnerabilities even after hours of reading through the code. Feed something like that to, say, Gemini Pro 3.1 and you'll get a bunch of false positives back, nit-picking, or variants of "this could be insecure if the code around it changes in unreasonable ways in the future".

Feed the same thing into GPT 5.5 x-high and then tens of minutes later it'll find half a dozen unauthenticated remote code execution vulnerabilities, arbitrary file read/write vulnerabilities, or similar.

Until it got nerfed, Mythos was similarly a huge step up for a lot of people working on code security.

Is this bug fixed with 5.6? If not, it probably doesn’t matter which version Codex users are getting because the overall result is dramatically worse than stated by Open AI advertising: https://github.com/openai/codex/issues/30364

> while you could ask your LLM to do that you aren't going to run your bank on the result.

Why not?

I feel like we're entering a new era of prejudice against not a category of humans, but against non-human intelligences.

The design patterns for distributed and fault-tolerant systems are well-known and established in the industry. Both humans and AIs are familiar with them!

So if you sketch a design for the AI to follow, establish the rules in AGENTS.md, have a robust test suite, use a frontier model dialed up to eleven, etc... why not rely on the LLM output?

At the end of the day, humans are not without fault either.

I've been wading through some legacy "pre-AI" code recently and it has more bugs than a rainforest! Static fields used incorrectly, causing data races. Floating point types used for money amounts. JavaScript and SQL injection up the wazoo. Wildly unsafe password handling. So on, and so forth. This is the norm for most human-written software, not the exception.

As a proof-of-concept, I tried an AI rewrite of one such legacy app[1], and it is not bug free, but it notably has fewer bugs than the original. Different bugs, sure, and I'll have to iron them out after a round or two of UAT, but I'm honestly more confident with what I got from the chatbot than the code inherited from humans.

[1] Deals with money, but admittedly at a much lower level of risk and consequence than a banking app running on a mainframe.

If this really is widespread and degrading performance in 40% of the cases, then if OpenAI simultaneously fixes this bug and releases GPT 5.6 within a day or two, then the sudden boost in capability is going to blow people's hair back.

Something I'd like to see is for someone to finally come to the realisation that the right thing to do is to make the front-end web templating language truly polyglot and support SQL natively, without an ORM wrapper.

For example, the ASP.NET Razor syntax allows HTML and C# code to be interspersed surprisingly freely:

    <ul>
    @foreach (var user in Model.Users)
    {
      <li>@user.Name</li>
    }
    </ul>

Just picture the same kind of thing, but with SQL expressions freely interspersed with the programming language.

Just like how Cargo, NuGet, NPM, etc... can import packages and/or how you can cross-reference projects in build systems, web apps should be able to reference a database schema project directly, importing the SQL definitions without any explicit "mapping". If the SQL changes, the type changes, and the build system picks that up automatically without any additional manual steps.

.NET with EF Core is almost there, and I've seen some half-hearted attempts in various languages over the years, but it's like the industry has an allergy to the concept.

Ur/Web is probably the closest to the idealised concept, and I think that's what I read years ago that put the dream in my mind: https://dl.acm.org/cms/attachment/feb131ab-37e1-4638-be17-ab...

I bet that since this bug has made headlines, there are some panicked engineers at OpenAI desperately trying to figure out how to fix it without undoing their “magic optimisation”.

Does this affect the Codex app too, or just the Codex CLI tool?

> I find it interesting how far we've come so far from the mindset of "You can do that."

Outside of Silicon Valley, most organisations strictly only buy commercial-off-the-shelf (CotS) software. If they do get something developed bespoke, it'll be an isolated thing with very defined boundaries.

This gets absurd after a point, where'd I've seen organisations was millions of dollars to avoid what amounts to an afternoon of scripting... and that was before LLMs!

If you even suggest custom code "an executable" (gasp!) to some people, they'll nearly faint from the stress of even thinking about it.

I've always found this kind of self-imposed constraint a bit odd, but it is surprisingly pervasive.

There can be selection pressure for females to mate with multiple males.

Releasing sterile males only works for species that mate only once or at most twice, and rapidly falls off in effectiveness for species where the females mate many times.

Also, almost any argument against LLM intelligence also applies to humans.

I very commonly see someone make some small mistake and end up going in the wrong direction, “accumulating stupid” as they go, sometimes for years.