plaidfuji

Haha exactly. I don’t think people appreciate how slow AI research was to start with. It was absolute drudgery.

Same is true of every other technology ever invented. Even medical advancements wielded by a malevolent power just serve to keep their army healthier. All of the other technologies you listed, in the “wrong hands”, contribute to the destruction of the world.

I think there are a couple of interesting scenarios where humans essentially “hand over the reins” to AI.

1. A leader rises to power who has instructed an AI “I want to control the world, tell me what to do”, and has some kind of real-time capable reasoning engine in their ear/head. Even this would require monumental advances in physical compute and HMI, but I wouldn’t rule this out within 20 years. However, this still requires a human to make the initial prompt and carry things out. And in such a scenario, there would be others with the same technology to counterbalance. I could see Elon Musk having this kind of idea. But even in this case, I don’t think “nuke everybody” would be the highest-ROI strategy advised by the AI.

2. An entire society hands over the reins of governance to AI. Basically some country decides that AI has advanced to the point that it makes better decisions than any human / group of humans, and allows it to govern their country with an objective of “take over the world”. This is probably closer to the imagined doomsday scenario, but again I find it hard to imagine that such an AI would advise nuking everybody.

If AI takes over the world, I imagine it will be because of the production efficiencies it enables. Which is a different kind of takeover - one that would eventually have broad benefits for many people - but would likely go through a period of major instability due to labor displacement.

As somebody who does ML research, LLMs certainly make ML research faster. Maybe even 10x in some cases, as the author suggests.

But that’s just the parts of ML research that take place within your compute environment, namely experiments on model architecture, training strategy, UI/UX, etc. The algorithms and software.

Just because that part of research is progressively accelerated does not imply that we’re on a collision course with world-ending “superintelligence”. It just means that you’ll get to the best possible model of your data faster. And even when your data is the entire corpus of textual knowledge, there is a limit to the utility that provides.

It just means that software engineering, data analysis, and other computational tasks will be less of a bottleneck in the overall progress of the world (which, by the way, is great, and extremely valuable).

I don’t think it’s a stretch to say that we’ll have reached max LLM capability within a couple of years (what the author refers to as “super intelligence”). But it is a huge logical leap to say this:

> If we go through this feedback loop, I expect that the AI systems that come out the other side will be able to very quickly develop novel weapons and very easily overpower human civilization in a conflict.

First of all, the weapons necessary to overpower human civilization exist. They aren’t used as such due to geopolitics and humans’ basic survival instinct. Second, developing novel weapons requires physical tests, which are decidedly not accelerated by LLMs.

AI is a tool wielded by humans. If we’re wiped out by AI, it will be at the direction of a human. But we’ve made it pretty far with some pretty unstable humans in command of planet-killer weapons. I’m not overly concerned.

> Given the election result the AI bailout ain’t coming

And why should it have? Ridiculous to pre-emptively talk about a bailout before a crash has even materialized. Who exactly was going to be bailed out?

What does this translate to in terms of number of 24/7 coding agents per engineering staff member?

I was going to ask, why would NK attack a China-based crypto firm? Then I learned that Binance isn’t headquartered anywhere. They’re literally asking to be hacked. Nobody will protect them.

This is also what keeps me using it. And whether this is rational or not, I feel that if it has a better / more concise understanding of core science/math, it also has a better shot at writing concise and correct code for those topics (a lot of what I do).

Even with the most advanced GPU-accelerated simulation packages, if you want to get to molecular-level detail and nanosecond-level dynamics (both of which are required for drug interaction), the compute time required for an accurate simulation of even a microscopic volume (say a cubic micron) is on the order of the heat death of the Sun. We’re so far from this it’s laughable. The best simulator is still the real atoms of the real world - we just need more reliable ways to control and measure them.

The discourse on tablets for kids tends to degenerate into a very judgy, black-and-white debate of so-called “tablet kids” vs… not, which I find really reductive.

Tolerance of tablet use is a spectrum ranging from the aggressively Luddite tech elite shielding their children from their own creation all the way to the 6-year-old posted up in the basin of a grocery cart dual-wielding their ruggedized tablet running Minecraft and their parents’ phone cycling through YouTube slop.

In my experience, most people fall somewhere in the middle, i.e. “we’re on our 4th hour in coach class, I’m tired of pretending this animal-themed card game is fun, and I know this isn’t development-maxing right now but we’re gonna do it for an hour to preserve our sanity and that of those around us”.

That being said, the problem with these ads is that they normalize tablet use as something that’s just another (even desirable!) part of everyday life, not a tool of last resort.

Not sure why you’re getting so strongly downvoted. The travel situations are the ones I find the most reasonable for tablet use. It’s definitely possible to keep a kid occupied for a 5 hour plane ride without a tablet but it takes work. Very few adults can raw-dog a 5 hour plane ride themselves, ffs. I may judge other parents for allowing tablet use in other situations, but not on an airplane.

> What I'm saying is the US broadly has low corruption. I don't have to pay a bribe to get my drivers license or to get through the airport.

Low compared to who? Third-world countries? Again, just shows how far we’ve fallen that these are the comparisons being made. The problem with the US isn’t direct low-level corruption, it’s systematized high-level corruption.

> Broadly speaking you can't purchase bills or laws, and have to convince the public.

There is a multi-billion-dollar lobbying industry whose purpose is essentially this, but through legal indirect channels. And the public are so ill-informed that spending money = convincing the public.

> The US has issues, but any supposed lack of data privacy laws isn't due to corruption, it's due to the will of the people

I tend to agree with this, but if there ever was broad momentum to strengthen data privacy laws, systematized corruption would stymie it. And is likely already actively suppressing it.

Also the funding of Trump Jr’s wedding by a Russian oligarch…

Private sponsorship of personal vanity projects (White House ballroom)…

Jet gifted by a foreign entity (Qatari AF1)…

Ambassadors demanding personal payment for guaranteed audiences (Guilfoyle)…

And that’s just in recent memory. The Left isn’t blameless either. The fix needs to be systemic (repeal Citizens U, re-build independent oversight and nerf the executive), but unfortunately I only see it becoming a deeper and more acrimonious blame game.

Slightly off topic, but I found this particular sentence interesting:

> This is software that isn’t ready for prime time, hasn’t been fully cooked, and is operating in a country that’s too corrupt to pass meaningful modern privacy laws or to regulate data brokers.

Something about the way this is worded, whether intentional or not, really drives home the reality of the US’ diminished standing as a trusted global partner. Like, we’re simply not taken seriously anymore. And it’s not just because of corruption - the corruption is there because of deep-seated political ignorance.

The author is a professor at Athens University.

If he truly believes this, why wait to stop thinking if AI is so capable? Why not have AI write all of your blog posts, papers, proposals, emails, conference presentations, letters of rec, etc? Read an AI script for your next keynote lecture. Every professor will be doing exactly that in the near future, as you predict.

Or maybe - maybe true critical and synthetic thinking will become even more rare and valuable than before. I actually find that when using AI in my job, I have to think more - I don’t get to do the slow work of crafting code anymore. I have to keep building my understanding of an ever-evolving codebase, on a cadence of hours. Because I have to explain what it’s doing at some point. I’m not sure that part goes away.

Or if I’m a professor writing grants or giving lectures, eventually I have to stand up and talk about it, and answer questions. So the machine can help me generate content, but I still have to internalize much of that content before I can confidently deliver it. So now that’s my rate-limiting step - the processing and digesting. I need to be doing that every day, and people will come to expect a new rate of output.

I think for some people, there will be an ability to tune out and become a manager of AI, just passing specs, requirements and feedback through. But I suspect that for many “knowledge workers”, the “knowledge” part is actually going to get more intense. That’s been my experience so far at least.

But maybe that’s just the last gasp of somebody being outcompeted by a thinking machine.

It’s a good question - my sense is it is a combination of factors. First, robotics hardware is not at the $20k price point I hypothesized, and the durability is also not good enough to realize a payoff. Second, enabling remote operation would require significant additional investment in sensors and probably enhanced control workstations on the operator end - unlikely to be doable from a laptop. A reliable high speed internet connection would also be required on both ends. And third, human operators can be kind of a wild card. They require training, management, and carry their own security and safety risks.

Again, I’m not saying we’re there, it just feels like the control software is the main bottleneck, and the development of that is now being accelerated by LLMs.

> understates the amount of real world (i.e. visual, physical, sensory) data and tacit knowledge

I agree the data trove for these is not nearly as comprehensive as “the entire internet” was for LLMs. But LLMs paved the way - people are building the datasets rapidly now. I suspect this will look a lot like LLM rollout looked over the past 5-ish years. Unusable garbage… less unusable garbage… then suddenly, a fully capable model like Opus 5.5 or the like.

And the “problem” with the way the foundational models seem to work is that they don’t just acquire individual skills one at a time like we do - they go from knowing nothing to knowing everything.

> I'm also continually surprised that people seem to think politics doesn't exist anymore. I find it exceedingly unlikely that massive amounts of people will simply accept this

This is exactly the part I’m afraid of. The political response of the displaced labor (which may or may not be violent / revolutionary), and the subsequent counter-response of capital… potentially using the very robots that displaced the jobs to begin with.

I generally agree with the prevailing commentary here, i.e. that automating app development and maths proofs may not be as world-changing as the Artificial Intelligentsia purport / “fear” it to be.

But also, upon more reflection, it occurred to me that if we’re able to create a competent humanoid robot, that could legitimately be world changing in some good and other very bad ways. The scary part is that I don’t believe there’s any major physical / energetic barrier to this. The hardware exists, the power supplies are light enough. The only part that’s lagging is the software and motion contols; the intelligence. The part that is experiencing breakthroughs at an accelerating pace.

What if a highly intelligent and physically competent robot were available for the price of a cheap car, say ~$20k? What if that robot were competent enough to cook, clean, assist the elderly, move things around, assemble and disassemble things, etc? Billions of jobs would be displaced, or hit with a very concrete maximum wage.

Many white collar jobs, while still at risk for displacement as well, benefit from barriers like IT integration and data privacy hurdles, and the fact that corporate higher-ups simply prefer to communicate with humans.

But most blue collar / manual labor jobs have almost the opposite condition: humans introduce risk, and generally don’t enhance anybody’s “experience” - see people’s somewhat odd acceptance of higher fares for a Waymo vs an Uber. [1]

I don’t think we’re there yet, but the fact that we are rapidly accelerating our trajectory toward that world should be at the least, unsettling.

[1] https://techcrunch.com/2025/06/12/waymo-rides-cost-more-than...

> If every exhibit must be scanned and uploaded for critical commentary by AI, so be it, but counsel may not supplant the record with an artificial facsimile and force the consumption of AI circumlocutions without end. If these sorts of filings pass muster here then they must pass muster in every other case. That development would mark the end of human-centered jurisprudence because it will prove necessary for judges and their staff to outsource their own duties to an AI tool.

I’m afraid this is where most cases of general “arbitration” are headed. Whether it’s a court case, you negotiating with your insurance / warranty / other hostile corporate party, or even job applications to an extent, it seems that there is an advantage to be gained by both parties using AI to overwhelm their “opposition” with generative babble. It’s almost like a proof-of-work hurdle that will need to be surmounted in order to escalate to a human.

I agree with this and whether it’s their intention or not, I think it’s for the best. People already become more callous in online interactions with other people, and that probably bleeds into everyday life as well. I would imagine having an AI “punching bag” might enable people to slip into the same behavior IRL.

We largely already missed our chance to stop the toxicity of social media - let’s not mess it up again with chat bots.

Wow I had never heard of this whistleblower guy Tyler, this email is honestly amazing.

> In my meetings with Daniel I found that the discrepancies between our CVs were due to Daniel calculating CV based on the median value of each precision run, while I was calculating CV of the entire data set for each level. When I asked him why we do this, he said that it was a way to average out the noise. I was under the impression that the coefficient of variation was meant to be, at least in part, a measure of how much noise exists in the data. By averaging out this noise before CV is calculated, the CV as a metric of assay performance becomes less meaningful.

“Becomes less meaningful”… this is the most polite way I can imagine telling your CEO, “hey, our results are all BS”. His message is so thorough and measured, and they just tear into him. He’s the only professional in the room.

Absolutely not. If an exec talks to you like this you should leave immediately.

I’ve said it before but the AI industry looks a lot like biotech / biomanufacturing. Huge R&D budget to produce products that are ultimately highly complex commodities. CapEx-intensive to build new “manufacturing” capacity. COGS matters. Regulation / IP control will be needed to protect innovators.

This is the correct answer. The pro-sumer / hobbyist / small developer market is tiny compared to enterprise. Enterprise will pay out of their eyeballs for a top-of-the-line product from a large US company. Doesn’t matter if the Chinese product is free.

Underrated joke

God I love Yann. All of the AI fear-mongering is perpetuated by the two companies that stand the most to gain from it: OpenAI and Anthropic. It builds an aura of mystique around their products to juice their valuation and stay relevant in the news cycle, and simultaneously builds a case to regulate their competitors out of the market. Even the people who have quit the companies over their “concerns” probably still have RSUs and stand to gain from the publicity, especially if they’ve pivoted into AI safety research. Easy to delude yourself when it happens to benefit you financially.

People need to stop the absurdity of imagining AI as some out of control independent entity. Every job is kicked off by someone’s prompt. Every job runs on models and compute owned by people. Assign accountability where it’s due: GPT didn’t hack huggingface - OpenAI did. They wrote the prompt, built the sandbox and ran the compute. When you write a program that hacks another company, you are responsible. This doesn’t magically change with LLMs. Also, if their model is so smart, why didn’t they use it to design the sandbox? Or was it incapable? Or were the humans too lazy?

If you build the world’s fastest train, start it up with no driver and don’t finish the tracks, when it crashes, it’s just your fault. Not the train’s. So OpenAI saying “we’re worried AI will wipe out humanity” is basically equivalent to them saying “we’re worried we will wipe out humanity”. Like, seriously? Don’t worry, we’ll take care of it if you even come close.

If there is a risk of that catastrophic of a fire, why would I rush into strapping one of these to my house?

Murakami has a very different style than Butler… unlike western novels there is less focus on plot and pace, but as you’re noticing there tends to be an ever-present mood that builds throughout. I couldn’t tell you a single plot point from 1Q84, but I remember the feeling of reading it very clearly.

I liked The Windup Bird Chronicle slightly more, and it’s an easier read than 1Q84 to get into him. He also does short stories that are phenomenal.

I worked in this industry for almost five years. Even the “fermented” alternative meats (bacterial, fungal cell culture) which were basically a drop-in fit with existing large scale fermentation plants were still not cost-competitive with chicken. Some were on par with beef if you made some hand-wavy assumptions about scale.

The killer was usually the downstream processing required to clean out off-flavors and colors, which was required to get even a passable product (these are just protein powder fillers at this point, not whole cuts), but ballooned CapEx and cut yield such that cost went back up to like, A5 Wagyu levels.

I’ve sat in the awkward tasting sessions the author describes, and they’re all the same. The attendees muster up a “that’s pretty good” - but we all know nobody would ever choose to buy this unless they were forced to.

Now that I’ve been out of it for a while, it’s pretty obvious to me what the problems are. Cost, obviously. Taste and texture simply aren’t there. And “the knowing” is real. I never got over imagining the whole factory process involved in the production. I’m sure vegans would say the same about animal farming, though. So my conclusion was, the only way this technology becomes relevant is if people are forced to use it. And that’s either because you’re cruelty-free by choice (small fraction of the population), or because the cost of meat suddenly skyrockets, which today, is not looking that likely. Keep in mind as well that meat costs would likely be driven by feed costs in such a scenario, which also impacts cultivated meat. So it’s ultimately a process efficiency play, and you’re competing against eons of evolution.

“… since we called for a slowdown in AI research”

But stating it plainly like this would make the contradiction too obvious.