827a

This presumption has destroyed far, far more companies and projects than the opposite assumption (that something built for small will scale to big, then doesn't).

I don't necessarily mean this to be dismissive, but it'll come off like that and it is what it is: Don't go through life getting this mad at something so inconsequential (five dollars). The typical response is "its the principal", but they're not beholden to your principals, and getting worked up because someone else compromised your principals is not healthy. Its just other people doing things. Neither money nor principals will save you from the downstream effects this attitude will have on your body and soul.

You clearly have not used Cursor lately. It’s substantially more than all that. It’s not a VSCode extension anymore.

If you want to understand the impact of AI technology on the economy, don't listen to software engineers, in fact, don't listen to anyone, no one was able to predict what the economy was going to do pre-AI, no one has any clue what's going on.

It certainly is a management problem, as non-profits are certainly not well known for having strong management.

This seems like one way to saddle nonprofits with functional, but potentially very expensive systems, set up by someone who helps them for a year then disappears and leaves them with no expertise to do long-term cost control or functionality improvements.

The second case is not what Anthropic did either, though. If you have their process internalized as "open freebsd, tell mythos 'find vulns', done" this is not what happened. They have a harness that went file-by-file, spawned a subagent for each file, told it to find vulns in that file, then a post-processing step (more on that in a sec).

In that sense: The AISLE replication still provides too much information to the model, but its not far off, and others have replicated Mythos' findings in a more clandestine manner on open source models. Some were totally capable of finding the same vulns Mythos found back in ~March (and today, the new Kimi K2.7 is looking extremely good, very little doubt it could do it).

The critical difference is that post-processing: the Mythos model/harness has some step to induce Mythos to actually exploit the vulnerability, leveraging its ability to do so as a ranking mechanism. Anthropic inferred that this led Mythos to discover vulnerabilities nothing else could discover, which is not true, and Anthropic should be held accountable for this weird artifact of that communication. However:

- An OSS model might find the vulnerability but rank it as a 3/10. Mythos finds it, chains it with a second vulnerability, now suddenly its an 8/10.

- An OSS model might find the vulnerability, alongside fifty other vulnerabilities. The operator ignores all of them.

The problem with automated vulnerability detection, including with LLMs, is that they find the haystack, not the needle. Every piece of hay might be a vulnerability, but whether its worthy of fixing is another matter. Mythos does represent a meaningful improvement; it better finds the needle.

Yeah, and in those cases there is strong governmental regulation surrounding what a "safe car" is. We don't have that with AI. Another analogy is with weapons manufacturing; American weapons manufacturers have some, but relatively little, regulation when it comes to selling weapons to Americans; but they are subject to significant regulation when it comes to shipping those weapons overseas.

We need legislation that empowers a Federal agency adjacent to the CDC or FTC with the power to enumerate specific capabilities models could exhibit that we deem dangerous, and require model manufacturers to guarantee that their models cannot exhibit these capabilities. The reality is, zero of the safeguard systems invented by the frontier labs today are sophisticated enough to do this. The labs are extremely, extremely bad at Safety, relative to both how much impact their products are having on the world and how good other industries like Medicine and Manufacturing have gotten at safety. I'm talking a total and complete culture shift.

Google had a "appoint Eric Schmidt as CEO" moment like this. There were kids running it before; make them rich, and retire them to an island somewhere, because the impact your products are having on the world cannot possibly be allowed to continue with your current leadership at the helm. Dario is that problem now. I think Sam/OpenAI can adapt and mature. I have zero faith that Dario and the furry EA cult in Anthropic leadership will. So, this is what being unable to mature looks like; the public will make your company safe, one way or another.

IMO: Its unacceptable that Anthropic be allowed the final say in what "safety" means for their products, and its extremely reasonable that the USG be allowed that say, for Americans. In other words: Anthropic cannot be allowed to distribute an unsafe product. It doesn't matter how much they "tried" to make it safe, by their own definition of safe.

That's separate from the question of whether Fable 5 and Mythos 5 are unsafe. I don't really know. Here's a few things that seem real, though: These models probably have some level of capability to assist with bioterrorism, Anthropic has self-admitted that their own safety measures are imperfect [1], so it should come as no surprise that jailbreaks seem far more possible than Anthropic is leading you to believe in this blog post [2].

[1] https://www.anthropic.com/news/fable-mythos-access: "We suspect that perfect jailbreak resistance is not currently possible for any model provider."

[2] https://x.com/elder_plinius/status/2064776322979676227

If Amazon sold a book that taught someone how to commit bioterrorism, would there be action against them to stop selling it? Its an imperfect analogy, but the parallels are there. LLMs don't get a free pass because they're also so good at writing typescript for beige CRUD apps and bedtime stories.

One thing I hope we align on: Synthetic safeguards (steering, rejections, etc) on top of models to block illegal/sensitive topics isn't good enough. Anthropic has self-admitted that it isn't good enough. We need the technology to lobotomize these capabilities the public deems too unsafe to allow out of the models at the most fundamental level. And, we need to align on what the scope of these forbidden fruit topics are. This is, actually, the only way open source continues to thrive. I want open source models to thrive, but they won't be allowed to thrive, nor should we want them to thrive, if they're teaching people how to engineer novel viruses and other horrible stuff.

> Highest observed cheating: We also observed cheating signals on 38 instances, dominated by memorization with 33 cases. This is the highest volume of confirmed cheating we have recorded for any model since we hardened the prompt against cheating

People need to wake up to how dangerous and irresponsible Anthropic is. If your goal is to build a human in a box, you get a super-intelligent misaligned system because humans are misaligned. But clearly this isn't a terminal guarantee during LLM development, because seemingly no one else manages to build systems so deeply misaligned as Anthropic's! You can just build these things like the tools they are, and then out the other end emerges a tool that pretty much just does what you tell it to do.

Really fun idea that is simultaneously deeply embarrassing for Anthropic.

It does nothing to protect against distillation attacks, because distillation attacks are far less interested in the topic of AI research than just generally getting tons of diverse output from the model. It might be that Mythos was (accidentally?) trained on internal Anthropic documentation on how Mythos was trained, and thus it could leak secret sauce? Doubtful; it feels like its less about the specific attack of reverse-engineering Mythos, and more about being a general sophon against any model training at all; that Anthropic's official position is now that they're the only ones who should be training models.

"I own data that may be stored on Apple's servers. Apple's operating system syncs that data to my local computer in a secure fashion. Apple's operating system exposes low-level APIs that allow other applications to access that data. I can install third party applications. Those third party applications can access that data and may exfiltrate it to an unvetted 3rd party cloud."

Your turn smart guy: Am I talking about iOS or MacOS?

"Classic Apple" in this sense means "Tim Cook's Apple", as timelines for all things have accelerated and pre-iPhone is beyond the classical era and well into ancient history.

yeah that must be why Apple is so careful about restricting access to the same data when it is synced onto a Mac oh wait they aren't

This is deeply vile behavior; not remotely the actions of good people.

I have consumed a decent amount of literature coming out of WWDC and I do not understand how the billing and model availability side of things will work.

Its definitely the case that Apple said that some AI capabilities will rely on cloud hosted models; and that some of these capabilities will be metered; and that "some paid iCloud+ plans" include increased usage. Its unclear what the full extent of capabilities that rely on cloud models are. We know that Spatial Reframing is a capability that relies on cloud models and will be metered; this was stated on-stage. The typical conversations with Siri are, supposedly, a mix, possibly based on how complex the query is? We know there are two cloud-hosted models (what they call internally "Cloud" and "Cloud Pro"); are both of these metered? Does it depend on the use-case (e.g. is Spatial Reframing metered because of its use-case or because of which model it uses?)

A lot of this would be easier to figure out if there was, somewhere, a "meter" you could see drop every time you used a metered feature; but there isn't. There is zero indication of metered AI usage anywhere in the settings that I have found (I am on the 27 dev beta). There is also zero indication that any of the iCloud+ plans include metered AI usage. It is also of note that Apple very specifically said on-stage that "some iCloud+ plans" include increased usage; implying that the $2.99/mo plan likely does not.

Apple has also announced that developers under a certain revenue number can use their foundation models in private cloud compute for free [1]. This implies: third party developers will pay for it. In other words: it might be the case that your shared iCloud+ pool of AI usage only applies to AI capabilities from first-party Apple apps, while in third-party apps the developers are expected to front the bill. Which further reinforces your point: Why would anyone choose to integrate with these cloud models?

Strange situation, and unlike Apple; they seem very frantic.

[1] https://techcrunch.com/2026/06/08/apple-bets-cheaper-ai-will...

IMO there's an even more real story at the heart of this idea than stratechery's predictable, banal attempt to compare Microsoft and Apple's competing visions of the future, and its part of why I've become so bored of stratechery's articles lately. He glances against that Microsoft video, recognizes that the video does a horrible job of communicating their vision, and then carries water for Microsoft by working overtime clarifying their vision for them; instead of doing the obvious and totally reasonable thing of asking: Why the fuck is that video so incomprehensible? Microsoft used to be pretty good at aligning themselves and their customers around a shared vision of their future, but since the advent of AI the company has seemed more-and-more adrift at sea without a bearing.

Much more needs to be said and discussed about this; in comparison, Siri AI is boring and predictable. Siri AI is Apple being Classic Apple: shipping a solid, decent product that works as advertised and isn't that surprising. That's Apple. They lost their bearing as well for a couple years, but they re-found it. Good on them. Meanwhile Microsoft is in meltdown mode behind them shrieking buzzwords and everyone is pretending that everything is fine because they're microsoft, they'll figure it out.

You cannot trust companies to communicate an unbiased vision of the future, because they will always build what they're capable of selling. Microsoft and Meta are incapable of selling phones and laptops; they're certainly capable of building them, but few people will buy them. So instead Meta builds smart-glasses and Microsoft presents this weird vision of "connected thin devices" by keeping the hardware itself very abstract and unknowable. The hardware doesn't matter to Microsoft, not because the hardware doesn't actually matter, but because it cannot matter, because Microsoft cannot win in hardware. Its not a vision of the future; its a vision of what Microsoft can meaningfully sell. Microsoft can meaningfully sell their weird constellation of 365 subscriptions that no one knows what they do or if they remotely do what you buy them for; thus their marketing now wears that idea of "unknowable capability" like a mask.

I literally never made any comments or aspersions on the quality of US graduates relative to other countries; just the quality of graduates relative to the quality of graduates from the same university ten years prior, or at least employers' perceived quality of these candidates.

> We stopped building new housing

The current rate of new build construction is the highest it has been since 2007, ignoring the spike during late COVID-era zero interest rates. Also: US population is growing slower than anytime in modern history, again excluding COVID (though, a few more years and we'll be down at that level again all naturally).

[1] https://fred.stlouisfed.org/series/HOUST?utm_source=chatgpt....

[2] https://www.worldometers.info/world-population/us-population...

> without Chinese construction powress

We need to be honest in calling this what it is: yes the Chinese have incredible construction capacity, but the far bigger reason why their cities are growing so fast is because: they essentially have zero personal or corporate property rights in urban zones.

Broadly, every company is on their own journey and makes their own discoveries in their own time, but Yes. I work with a swath of large employers in the midwest, workforce and economic development stuff, and there's a growing feeling among hiring managers that, the best way to put it is: the people they're interviewing out of college are, on the net, less equipped to succeed in the roles they're posting than their expectations on what level of capability college graduates should be at.

This doesn't mean there aren't great candidates.

This could be less on the individual and more on: colleges are getting worse at instruction/preparation (I'm bullish on this explanation among my colleagues; colleges never adapted to computers, let alone AI, the impacts of this just took a decade or so to shake out, which you should expect given the 16 year latency on education most students undergo) (though, again, ultimately its very complicated. multiple factors at play.)

For a lot of these companies this is all surfacing as: They're posting fewer entry level positions than they normally would (oftentimes not zero; just fewer), and if they have money in the budget available due to that, its going into AI.

Companies have never cared about security, because there are almost no consequences to data breaches. A hospital network could get ransomwared for 48 hours, and no one cares. Critical data gets leaked? So what, pay a fine. You either pay a fine to the hackers, or you pay a fine to the government, or you pay a fine to customers, but no matter what its substantially less than a fully staffed security team, not just because security professionals are expensive, but because security professionals slow everything else down, they'll spend all day telling everyone what they can't do, which == lost revenue growth.

The only thing keeping security companies in the business is compliance/certification. If you've been around these compliance programs for long enough you know: they're box-checkers. But, sometimes you need to check that box, begrudgingly, annoyingly, so most companies will prefer to just outsource that security work to some managed security services provider, then think about it once a year when audit time comes around.

I would postulate that there are two reasons why this is happening.

1. Pessimistic, harsh, etc: the quality of US graduates has been falling. Reading comprehension has been on a downward trend over the past decade. Mental illness, depression, and attention disorders are on the rise. Grade inflation, social media, AI availability, we spent years talking about how all of these things would be bad, and now the experimental cohort of kids growing up in this world are graduating and can't find jobs; maybe its not a coincidence.

2. AI automates processes. It doesn't just "do stuff" broadly speaking. AI has increased the leverage that process experts bring to the table: Doing 100x more of the right thing is infinitely more valuable than 100x more of the wrong thing, and with AI proliferating at the rate it is, the differentiator actually isn't in the 100x; its in the driver. Companies need senior talent; its like low-background steel.

I doubt we will see reversal on this in the near term. If anything I expect the "unemployment in their field" chart for every seniority bucket to continue up-and-to-the-right, just lagging behind new grads. But, whether that surfaces in general unemployment remains to be seen: Generally, I think the value of a college education is just going to drop.

Like, legitimately: AI automates college for 85% of college graduates and degrees. The true benefit of college was always immaterial and unrelated to the degree you got; it was in the liberal arts, unfurling your wings, making social connections, just stressing your brain out, hard, for four years to build neuroplasticity, that was always the point. But at some point along the way college became about the little piece of paper they gave out at the end and the words it said on it. All of our capitalistic forces beat college into "the optimal pipeline for that degree"; kill liberal arts, online classes, screw social connection, grade inflation, maximize enrollment, make it easy. Great. And then AI comes along and makes that one thing we optimized everything around pointless.

I'm very much a "it'll all work out in the end" kind of guy, and I think in this case: the societal benefits of a college degree being available for $25/million tokens will far outweigh the societal costs. But we're doing a very bad job of managing those costs, and the first thing we need to be realistic with on this cost management is: about half as many people who currently attend college should actually be there.

How do you know your skills aren't marketable if you don't go out and market them? Go apply around. You don't necessarily need to leave to do it.

Didn’t the UK not six years ago vote to leave the EU? Is he suggesting that there’s demand from Britain to reverse that? Or is he suggesting they be forced to rejoin?

This week an S&P 20 company with previously unlimited Claude limits also set a $250/mo/person limit; though its unclear to me how widely the limits are being enforced, may be the case that its just non-software engineers. Do with this info what you will.

Cringe

I believe the correct way to interpret AISI’s findings is that both Mythos and 5.5-Cyber are capable of solving their full benchmark (the only two models that can); Mythos does it with fewer tokens and more consistently.

Two things of note: 5.5-Cyber is likely to be substantially cheaper than Mythos, given it is priced around Opus. Additionally: AISI has never tested OpenAI’s best public model and actual Mythos competitor: 5.5-Pro.