fwlr

Great way to inject some price information into the “AI automates exploits at scale” narrative. “AI can now chain complex exploits, it’s point-and-click pwnage, you say? Over or under $500 a pop?”

If the actual price tag behind the “AI can trivially automate exploits, be afraid” narrative is more like $15,000 and not like $150, that puts the narrative in a very different light.

Well, of course not. If you borrow a trillion dollars to build a 1¢ aspirin machine to compete with $1 generic aspirin, you are strongly incentivized to sell your aspirin for 99¢. We don’t see productivity gains, because approximately all of the gain is captured as revenue for AI companies.

It’s weird that this has to be spelled out for so many engineers and economists, but to be very clear: OpenAI et al wants your token spend to go to one-minus-epsilon of what you previously spent on payroll, and wants your productivity to go to one-plus-epsilon of what it previously was. If you want 10x productivity, spend 9.9x more on tokens.

They are in the business of enclosure and rent extraction, and any innovation that comes out of their research and development - regardless of how real an innovation it is - will be completely contained and bent totally towards that business. Any leakage of the innovation that you experience as a productivity boost is literally felt as lost profits by them.

I upvoted it while I ruminated on what I wanted write as a comment. It could have been a “multiple people are typing…” moment.

  Once a robot can do everything an IQ 80 human can do, only better and cheaper, there will be no reason to employ IQ 80 humans. Once a robot can do everything an IQ 120 human can do, only better and cheaper, there will be no reason to employ IQ 120 humans. Once a robot can do everything an IQ 180 human can do, only better and cheaper, there will be no reason to employ humans at all, in the unlikely scenario that there are any left by that point.

Machines are making great progress on the IQ 180 front, and now with this it looks like they’ve opened a beachhead from the other direction too. Classic pincer move.

  In the earlier stages of the process, capitalism becomes more and more uncoupled from its previous job as an optimizer for human values. Now most humans are totally locked out of the group whose values capitalism optimizes for. They have no value to contribute as workers – and since in the absence of a spectacular social safety net it’s unclear how they would have much money – they have no value as customers either. Capitalism has passed them by. As the segment of humans who can be outcompeted by robots increases, capitalism passes by more and more people until eventually it locks out the human race entirely, once again in the vanishingly unlikely scenario that we are still around.

https://www.slatestarcodexabridged.com/Meditations-On-Moloch

  There is no emperor distributing boons for the mathematicians of today.

Yes there is. The owners of the models will be the emperor, and we have already seen some suggestion for how they will distribute boons - consider OpenAI’s Navier Stokes result and how they offered co-authorship on condition of excluding the employee of a competitor.

There is also a subtle fatalistic fallacy at play in the author’s claims: “Progress is inevitable. This is progress. Therefore this is inevitable.”

But this doesn’t follow. We may reject this instance of progress, and allow the inevitability of progress to furnish us with another option.

Not to worry. The customer can shrug and say “rogue agents took unaligned actions”, and they too will be absolved of liability.

Right?

I’ll have to check these out! Thank you!

If you can tolerate the in-ear physical sensation, Etymotic makes some fantastic earplugs for reducing noise. I similarly found even small noises to be distracting, and I couldn’t avail myself of earplugs or noise cancelling headphones because they also felt distracting. The clever thing that Etymotic did is design to reduce noise evenly across the spectrum, so it feels like turning the world’s volume knob down, rather than unevenly suppressing certain frequencies. Noise seems qualitatively less distracting with “the volume turned down” in this way, and the uneven suppression turned out to be the part that bothered me about other options. I consider myself sound-sensitive yet these things made a 14-hour flight in the same row as a crying baby quite tolerable (to both mine and the suffering parents’ relief). Your mileage may of course vary but this was a big QoL upgrade for me and it may be for you too.

Looks like an exponential curve, the inflection point doesn’t seem to line up with either the introduction of AI, or AI’s inflection point. Yes, yes, lower your prompt box, Software Development Has Changed Forever In The AI Era, but this is just “CVEs got Goodharted”.

As long as you are aware that it will favorably compete for your time against work and sleep, you should be okay. It sucks the same juice that programming does, so it might also be a good idea to schedule a week off work for it. At around 1200 hours, all of that several years ago (I am still saving Space Age for a future break) I do consider it to have enriched my life.

Investment isn’t debt, but investment and debt demand revenue in functionally identical ways, and OpenAI has taken an enormous amount of investment, so they are subject to broadly the same pressures as a hypothetical company that had borrowed lots, no?

Are you sure you aren’t looking at a corporation’s revenue chart?

“Prisoners say prisoner’s dilemma is bad yet defection rates have never been higher”

Music before quantization:

                soul | soulless
    listenable ______|_________
  unlistenable       |

Music after quantization:

                     | soulless
    listenable       |_________

Hmm, this “net improvement” of yours seems to have obliterated some important categories. I don’t think I care for it.

…the pre-existing staff, who by virtue of being staff of the intelligence agency, would be exactly the humans who are permitted to use these gated intelligences, and can therefore use them to explain the results, as described in point 5 of my previous comment? I’m not certain I understand your question, is that what you meant? Or did you mean something else?

In any case, it was just one example, I am sure we can think of plentiful others.

I think you may be interpreting the author’s self-deprecating style and gallows humour a little too literally. I took “I have not read the paper” to mean “I have not been able to understand the paper by reading it”, or perhaps “I read the paper but could not follow it”. There’s no doubt that every word on the page did at some point enter the author’s eyeballs.

Why should we expect this? Does it help the corporation’s bottom line? How much? Is that worth the R&D spend and compute budget it would need? What advantage do they stand to gain by demystifying their product, that they don’t already get from e.g. Lean verification? Do they gain more if the same research and compute spend on their inscrutable intelligence is directed into making it more intelligent, instead of less inscrutable?

I mean Patch the Planet. This was the first I had heard of it, and I assumed from seeing the press releases from OAI working their way through journalism’s digestive tract that it was announced today - but perhaps it was previously announced.

If that is case, I’ll refine my statement to “Also happening today: OpenAI is doing a PR push on their software vulnerability initiative”, which I think suffices to convey the same meaning as the original.

Why does this not feel like a healthy place to be?

The author of those sentences is also the author of the main result that this generated improvement builds on, and prior to this improvement his work was thought to be unimprovable. He is, in all of the important ways, literally the single most qualified human on the planet to asses this result. And his assessment is 1) this improvement is real, 2) this improvement is exponentially bigger than his work and the work he improved on, 3) it is effectively impossible for him (and by implication, humanity) to understand the result as presented, 4) it is seemingly trivial for “frontier circa 2026”-class models to understand the result as presented, and 5) to the extent that he is able to understand it, it is only with significant frontier model assistance.

So the place we are at, at least for this highly intellectual field, is “intelligence is the property of ~2 corporations, you must pay them a fee and agree to their terms and conditions to access it”.

At risk of stating the obvious, this is not a healthy place to be because intelligence is now structurally subordinate to the profit motives of particular corporations and at the mercy of all the attendant economic/sociopolitical incentives that apply to them. If the landscape is such that it is in these corporations interests to withhold this intelligence from some and make it available to others, that is simply what will happen.

One trivial example, of which there are an arbitrary number: An elderly leader is convinced to issue a decree that all number-math-computation intelligence is to be made available only to the intelligence agencies of his nation, and failure to comply means no IPO and no further investment. From this point on there would be a permanent nation-state-shaped shadow over practically all modern cryptography, with no recourse possible.

If the Church had a Large Astronomy Model in the time of Copernicus, the earth would have stood still - regardless of what it may privately prove about heliocentrism, it would never declare e pur si muove.

I agree. At present I do not see any set of incentives, meta-incentives, etc., that are extant, or even rigorously imaginable[0], that will lead to such a state of affairs.

0: credit to Xenosystems for this excellent phrase

All those mature charting packages are solidly inside the LLM’s training set, I’m a little surprised it can’t do a better job of reproducing them.

IMO it is a little bit silly to describe “a system that works well until it intersects with a trillion dollar company’s PR campaign” as an outdated failure.

> why is it being proclaimed the worst vulnerability in years?

Also announced today: OpenAI’s software vulnerability initiative.

Eh, then it just shows relative change between results; in isolation, yes, not contextualising these results at all is better than misrepresenting them, but doing both is arguably worse than either, it kind of admits you know exactly what to avoid saying.

I’d like to nominate the graph there for some sort of crime, for using a log scale from 1e-55 to 1, where there is one result at 1e-55, one result at 1e-18, a million results at 1e-3, the omgwtfbbq threshold drawn at 1… and each result is drawn as a bubble that covers 3 orders of magnitude above and below.

My understanding at the time I wrote that was “someone else using chat-logged OAI model: September???; OAI using unspecified model: October???”, and when I say “Navier-Stokes-style-scoop” I am thinking along the same lines as the comment I made during that controversy, here: https://news.ycombinator.com/item?id=49639648. I don’t buy that their public statements are not “technically correct but misleading without internal context” on these matters.

Naturally, with the added info from the PHP foundation link (specifically that OAI reported it the same day as Calif, and that Calif was operating under the auspices of an OAI program) the timeline collapses for my initial impression!

So from what I gather, it was found by OpenAI (via unspecified model?) and independently by Wiz (via Gemini Argon)… but before that it was found by Calif (via unspecified but consumer-available model)… but Calif credit LCFR_HAX before them with first discovery, disclosure, and remediation efforts.

I guess either LCFR or Calif put it in the training data (by using a consumer-available frontier model in the process of discovery or remediation), and then OpenAI reproduced it from the training data, Navier-Stokes-scoop-style? Except I guess more plausibly unaware, and also late? They thought they had something novel, because they weren’t privy to the existing remediation effort? It’s all rather confusing, and then on top there’s a leak and a PR campaign and it just becomes impossible to untangle.

edit: thank you PHP foundation for making this all make sense https://thephp.foundation/blog/2026/10/21/todays-php-securit...

I can literally feel my eyes physically defocusing trying to read this, and I can’t help but conclude that, at a level below conscious thought, my brain is trying to defend itself from AI-generated text. Perhaps it is more of a cognitopoison than I explicitly knew.

What is not self-aware about criticising machines for stealing the work that you were previously doing?