10,248 karma · joined July 16, 2012
If anything I've written on this site seems interesting, or confusing, or you think I'd be interested in something you've written/read, please let me know: hn@justinblank.com.
A chief scientist can be super influential, or a guy who’s on the slow path to retirement but is keeping a paycheck to keep up appearances.
By that standard, this guy is doing quite well. It's a good post.
They announced it publicly within days. https://huggingface.co/blog/security-incident-july-2026
Your job is to maintain and modernize a system while delivering a constant stream of new features. Your system is several million lines of code, with some multi-thousand line classes, a rulesengine that can trigger nearly unlimited effects at any time, a persistence system that's weirder than anything any of your friends have ever worked with, and hundreds of customers delivering tens of millions in revenue who use the system in incredibly varied ways.
You can't stop to rewrite the thing, you can't just throw features out there and pray, because you'll cause regressions and your existing customers will hate you. You have to fix the thing as you're building on top of it.
But where I agree is that there's no single deep concept that unlocks it all, it's not like you'll fix it by reading a textbook about it. It's complicated, and it's going to stay complicated, no matter how long you work on it.
I don't know how to say this in a way that won't sound insulting, but I don't mean it to be insulting. Programming, even systems engineering, is a surprisingly shallow field.
I don't mean that it's easy--it's not, it can be incredibly difficult. Difficult and deep are just different concepts. Difficult refers to how challenged you are. Depth, at least as it appears in math, is closer to a structure where concepts build on each other so that if you don't understand one concept, you can't understand further ones.
Programming can be difficult, and it can be intricate, but it is rarely deep in this fashion. Being deep in this way isn't the most important thing.
If I go into an area of programming that I don't have a lot of experience in (graphics, or the linux desktop environment), I will not be particularly useful, and it will not be easy. But I'll not experience the same type of impenetrability I experience when I try to read a paper on topos theory.
One more way of putting it: people are giving the example of TCP. You can spend a decade learning about TCP (or SQL semantics, or web standards). But what is happening is that you're filling in gaps in your knowledge. Meanwhile, in math, you do four years of undergrad, and even if you're a strong student at a typical university, there are topics that are still years away from you being able to touch them.
Computer science is a mix. Parts are deep, parts are shallow. Parts just are math. The odds that I can read a dissertation in computer science are decent. For math, they're much much much worse.
Like, they don't say "hey Sol, here's the password to SamA's bank account."
The premise seems to be that models aren't smart enough to understand this, and if they were, they'd be sentient and want autonomy.
For an article that's about making things up, and being too trusting, this seems bad. Maybe the author knows a lot about LLMs, but it doesn't seem like it.
Pasting the verbatim quote from the article into a free ChatGPT session: https://chatgpt.com/s/t_6a5e6f5e24f08191b6a482aad63cae63
Going to an incognito window and using a less leading question: https://chatgpt.com/s/t_6a5e6ee4a3508191bc1b352b41911b53.
If I go generic and just ask if there's anywhere I shouldn't drive, it doesn't get to Lunar New Year until I ask about "events" on the third question: https://chatgpt.com/s/t_6a5e6fb239208191b18cebcf7642c8b0. It's sort of a win for the article, if you think that people who run driverless car companies are all dumb, and won't create a prompt to tell their LLM to "consider events that might disrupt traffic."
I don't think a content creator would naturally describe themselves as a content creator. The word is useful for contexts where you need to generalize over all the different types, but that's not useful when speaking about yourself. So a writer says "I'm a writer", not "I'm a content creator." But writing this paragraph, I needed a word to generalize.
P.S. The closest alternative word might be artist, but it's not quite right--some writers are artists and some aren't.
Billions of dollars of annual revenue go through Claude Code, and the people who work on it must be a lot of millions in headcount.
The time matters a fair bit, it's probably time someone can't spend breaking things in the name of new features, but if rewriting the stack had even a tiny impact on retention or driving higher usage, it would pay back that $168k.
Why do you think this would happen? There's no mechanism that makes young gen objects that reference old gen objects (or are referenced by old gen objects) get promoted faster. You have to survive a certain number of collections.
2. Those people will say whether it's a good proof or not. We have other examples of interesting proofs from AI, we're really beyond the point of arguing whether it can produce any interesting math (though it seems to do much better at combinatorics than anything else).
I disagree. Mathematicians care about the utility of a result. It is just that they regard mathematical understanding as a valid type of utility, and that can be arbitrarily far removed from practical utility. But a proof that doesn't help anyone understand anything interesting is not valued. I could go out and define some pointless construction and create proofs about it immediately. It would only matter if I connect it to some other subject of interest within math.
I would argue that mathematical understanding is valuable for extrinsic reasons, but it is true that by the time you're a math grad student, you're usually willing to pursue it for no external purpose.
Although not a mathematician, Daniel Dennett had a wonderful example about higher order truths of "chmess". https://personal.lse.ac.uk/robert49/teaching/ph445/notes/den...
If I criticize your code, that is a professional criticism.
If I criticize your code and say it reflects your consistent carelessness and stupidity, it is also personal.
If I say you fabricated something, then that is a personal criticism, it alleges an ethical violation. In a professional context, it's also a professional criticism (every profession has some ethical standards).
Of course, that power also makes it harder to anticipate security issues--if you can't solve prompt injection, you have to reason as if every thing you allow the LLM to see is an API that an attacker has access to.
However, there are still necessarily going to be middle points where the LLM is more capable than a menu.
Now, on the first order point, I agree that non-tech companies seem to be taking longer to see results from AI, even if the argument was bad.
I work on SaaS for the logistics space, and I feel like prior to the end of 2025, almost all the discussion about AI for logistics was vaporware, starting this year, companies are actually trying to deploy agents, and we'll start finding out what the ROI is later this year or next.
I'm in the pagerduty lineup, and shockingly, my IQ isn't even a mere 185.
Many things about the process are similar, so there's some analogy, but it just isn't the same.
Such a weird thing to start with. The legal status of Fable does not mean that it's not intelligent. If anything, the problem is the opposite, someone thinks it's too intelligent (and/or that Anthropic wouldn't share its last gen intelligent models on the terms the government demanded).
It's true that LLMs do something that looks very compression like in their weights, but it is lossy, and it has to be--if you're not lossy, you've overfitted the corpus, and that's bad. Post-training takes this even further, because you're not doing anything that looks like training on a specific corpus, you're exploring in a wider space of text. That text doesn't even concretely exist until you start exploring it.
I'm sure there must be a serious attempt to pursue this analogy that isn't just handwaving, but I haven't seen it.
Perhaps the closest is DeepSeek v4:
> Hyperpape is a user on the LessWrong forum, known for thoughtful comments on rationality and philosophy.
I studied philosophy, so maybe, except I don't post on LessWrong, and I'm not a rationalist.
The law seems...truthy, though I find it a little too underspecified to assess.
The conclusion is true, and I'll even overlook it being a slight strawman. Your "long genius"[0] CEO cannot talk to AI and get a full business overnight. That's true.
But how does that follow from the "law"? The article admits that you can shift complexity to a complex algorithm or information processing system that the consumer doesn't touch.
You know what's a incredibly complex information processing system? GenAI.
So we have a reasonable conclusion, an ok "law", and no real connection between them. It was an inane non-sequitur.
[0] Not actually a genius.
> In my Ottawa life, every Tuesday evening, I take two gym classes back to back—boxing and the pompously named “body sculpt,” which makes me discover muscles I didn’t know I had.
The em-dash matches how you'd speak out loud.
You'd say "I take two classes every Tuesday back to back, boxing and 'body sculpt'. Weird name." (Parts of that sentence did flow oddly, but not because of the em-dash).
Grammarians say you can't make those separate sentences without adding some extra words, and because of blah-de-blah-blah-blah, someone might say you can't join them with a comma. So we have an em-dash.
Rewriting the sentence would make it flow less naturally, not more.
In exchange, it also less secure, less user friendly, and less popular.