jchw

(2023)

You have to realize by now that outside of academia nobody is even paying any attention to people who are predicting existential risk that far out. For worse or even worse, AI 2027 is the AI safety conversation we're having today, and I regard AI 2027 as essentially Anthropic fanfiction, except usually fanfiction is written better.

In spite of this, I actually kind of did build a "software factory". The main reason why was because there is a number of things I build that really are just means to an end, tools where I could not give a shit less if the code sucks or anything as long as it does roughly what I want. This stuff will most likely never be posted anywhere, but that's not the point, because it still has quite a bit of utility. Having a "software factory" to build and maintain it seemed a lot less annoying than being a "meat proxy" all day just to maintain some tools that aren't very important.

I could be misunderstanding what a "software factory" is but I think whatever I have done fits the criteria. I have a custom agent that goes around implementing software and maintaining private repositories completely autonomously and I can communicate with it using issues and comments.

It seems like a useful pattern, just maybe not what people had in mind. I think people figured that you'd build an AI Microsoft or something. I don't think that's true because most of what a company like that does isn't really software engineering anyway, but there is probably still something to this pattern.

They asked:

> Can you provide examples outside of gross negligence on part of the dog owner?

And the answer is no, I can't. The reason why is because there is essentially no such case. It is basically negligence by definition when your pet hurts someone who didn't cause it directly themselves because it is your responsibility.

So I asked:

> Can you provide an example of where your dog harming someone else is in fact not negligence on your part?

Which is a really easy question to answer and it seems you could not.

They certainly didn't invent this strategy, but that's because it already existed. Regulatory capture is nothing new.

> 1. They, and many other experts, have been saying the same thing since well before the current paradigm existed.

Some experts sure. "Some experts" say lots of things.

However, this is actually much to their advantage. If you were a frontier AI company, a legitimate concern to use as justification for your regulatory capture is exactly what you want. That is, after all, generally how regulatory capture gains public approval.

AI safety was also why OpenAI had to charge per-token for GPT-3. It must've been such a struggle for them to make that difficult decision. I'm sure Sam Altman spent a lot of nights crying to sleep over that one.

It's good that we never got open weights for a GPT-3 class model, right?

> 2. There is nothing about regulation that is required to give them an advantage; it’s a fallacy to suggest “regulation = advantage”

It's difficult to outline the number of different ways that regulatory capture can and will benefit OpenAI and Anthropic.

If we limit things to "Fable"-class models, then nobody competing with Anthropic/OpenAI is ever allowed to outdo them. If you literally legally aren't allowed to make better models, you can never charge the "premium" of having the current top-tier model. You have to compete by being cheaper than Anthropic instead, which is a lot less enticing from a standpoint of trying to become profitable. Take a look at the per-token pricing of Fable real quick. Yeah. Being able to charge a premium for the model that does the cybersecurity thing is literally their most important asset.

If Anthropic really is making a profit now as rumored, then they need regulatory capture as fast as they can possibly get it. Once Anthropic and OpenAI prove that AI is a viable investment, entities with a lot of capital will suddenly become even more interested in re-allocating it towards taking a slice of this pie.

> 3. It would be preferable to argue the facts (is it or is it not dangerous? Why or why not?) than conspire as to why they’re stating what they and plenty of others have been saying for decades

OK. On the factual end of things, AI safety experts have been raising the alarms for over a decade and so far essentially none of their predictions have come true.

The rationalists, the main group pushing the AI safety rhetoric that dominates the Western AI sphere and is virtually absent in China, which appear to be going entirely off the deep end at this point in time, put together their best minds to produce AI 2027, which of course like most predictions started to fall apart a bit before hitting the one year mark. Predicting the future is hard.

I have made this point before but I will make it again: I think AI safety experts deserve at least as much skepticism as Ed Zitron in their claims. They've been making similar dubious claims about AI capabilities and an incoming singularity for a long time now, and so far the best we can say is that maybe it will eventually come true.

Maybe. Or maybe you've read too much goddamn science fiction.

Go also just produces suboptimal code because it isn't maximally geared towards runtime performance. It lacks a lot of the more expensive optimization passes, like auto-vectorization.

On the other hand, time and time again it has been proven that it can be a good language to write tools that are perfectly sufficiently fast, like esbuild. With ts-go the choice to target Go instead of Rust seems to have come down to sensibilities and ease of translation, and possibly some other factors as well. I don't think they were aiming for maximum performance, and it would be unsurprising if you could squeeze quite a bit more out of ts-go itself with some effort.

The real benefit was getting the initial "10x" improvement, of course. Getting an additional 2x-3x is fantastic, but it's less noticeable when your codebase is already getting typechecked faster than you care to measure. For me even my largest codebases are chewed through plenty fast on ts-go, and I trust the TypeScript team to have done a good job (especially having seen some of their process for doing the port) so I think I'm good on missing out vibecoded Rust ports and whatnot. Interesting for sure, but we'll see how it all pans out in the long run.

Can you provide an example of where your dog harming someone else is in fact not negligence on your part?

But negligence seems like exactly what OpenAI and/or their contractors are guilty of anyhow.

It's a program on their computer that they ran on purpose... And yes you are definitely criminally liable for things your dog does.

The agents are clearly not accidentally compromising systems. Seems very easy to trace back the responsible parties here, I really don't put much stock into this argument.

I think it may just be bad automated flagging. I vouch'd them since they don't really seem to violate any rules/guidelines (right?)

I don't use OpenCode but I would like to know who this impacts. Surely people who are just using OpenCode locally with their own credentials wouldn't be impacted, right? This sounds like something where you have a hosted service and you put your keys into that. Not saying this isn't bad, but it isn't clear and they're not sharing any information.

There is no obvious explanation for why you would want this anywhere I can find... Does anyone understand? If you wanted this, wouldn't it make more sense to extend the last version of TypeScript before ts-go instead? Hmm.

Well no, you probably wouldn't get thrown in jail for that for the same reason that having your car stolen and used to commit a murder wouldn't usually get you thrown in jail. I dunno what you mean by suggesting crimes committed using computers are ambiguous. I mean the CFAA is vague and all, but more in an overly broad sense, which in this case seems to make it even less ambiguous.

Sure, sure, but they've already won by even having a policy against AI slop articles and posts, which, to my knowledge, HN seems steadfast in refusing to do so. After all, the public is fiercely divided into mostly one side in favor of banning said articles. HN is apparently just going to ride this out with only a policy banning LLM comments.

At this point, my only reason for still being around on here is because I simply don't know of any place that I find strictly better. Most of the popular alternatives to HN are annoying in ways that I find equally hard or even harder to tolerate.

It's amazing and at this point, absolutely no surprise that HN has worse moderation of genAI than the cpp subreddit. The conflict of interest with AI has hit HN like 9/11.

My main problem with modern Apple software is I feel the design has been on a near constant gradual decline trend overall (except for the huge dip with Liquid Ass of course) and it has gotten buggier too. I'm just never going to update my iPad because I updated my other Apple devices and I deeply regret it.

I actually have felt this way for a fairly long time but Apple always baits me. Like the twelve or fifteen different times Apple OSes have finally supported some parts of WebM on certain devices sometimes when it works. Oh my god, I gotta update so I can have another couple WebMs load successfully only in Safari and not in any other app that doesn't decode it itself. Or back when I used an iPhone, third party keyboards - I use a lot of jailbreak tweaks but I gotta give that up so I can upgrade for third-party keyboard support that is so ass that nobody would ever use it. Heck, I wish I had updated my iPad even less. One fewer major version and I could've made use of the virtualization support that was dormant - but Apple fixed the glitch that let you use it on Apple Silicon iPads, so now you can't. Thanks! This is important to do because it helps... Uhh, security, probably? Something like that.

Since Snow Leopard or so, most of my Apple software upgrades have been bitter sweet at best, but usually just bitter, like I've been baited into having a worse experience.

Of course, it is probably apparent that I simply don't like Apple very much, especially not anymore. And I hate to have to justify myself by doing this, but we lack the maturity needed for me to avoid it, so... I am going to point out the obvious that this is hardly limited to Apple. GNOME 3 may be the single biggest downgrade any desktop system I've ever used has been. It has improved over time, but GNOME 3 has never been even close to as relatively good for its time as GNOME 2 was. Same for KDE 3 -> KDE 4 although I think as a rare standout KDE Plasma 6 is genuinely getting better continuously over time and remains one of the things I am excited to update. Windows 10 -> Windows 11 was so bad that it seemingly durably made several of my friends switch to Linux full time and just simply give up on things that absolutely can't be made to work on Linux - that is a powerful statement, and months later it seems they were indeed not fucking around. Android has been enshittifying at an alarming pace, too, although I don't feel like the Android experience from a software perspective is degrading or getting worse, but Google really knows how to blow their own leg off and by god they're going to do everything they can to make sure that all of the advantages they have over Apple are rendered meaningless so they can drag themselves kicking and screaming out of accidentally having succeeded at something.

I don't feel like it should be necessary for me to justify this, because Apple's recent growing incompetence in shipping working software and release engineering stands on its own regardless of industry trends. I'd SSH into my Mac Mini, but I can't. I have to go reset it, because it's been on for 60 days and can no longer open new network sockets. (I am not making this up. macOS genuinely has/had that bug and mine genuinely is in that state right now.) It's not that everyone else is doing great. When it comes to Linux systems it's easy to give them a lot of slack because they simply have no goddamned money, but still, things could be better. It's just that Apple is richer than ever and yet worse than ever at shipping software. I struggle to square the circle.

I really wonder why I think the N Vision 74 looks so much better; it just has more personality somehow. I do agree though that the new Roadster looks a lot less bad than the Cybertruck.

I am not sure what you mean to say with this but reading back through this thread I'm not sure if what is implied here has come through very well. The idea is that well outside the range of where people died from being evaporated, there was a much larger range of people who were exposed to a lot of ionizing radiation from various sources like black rain. But, then, there's a range where people exposed to relatively low doses but higher than background radiation seem to have lower cancer incidence than average. So there is a distance where it is higher than average, then further away from that, it becomes lower than average, before finally going back to average.

This doesn't necessarily imply that radiation hormesis is the only possible explanation, but the easy hand-waves are all basically out. Something weird is happening here and it seems to repeat in different circumstances, too. There is not much to go on other than confounding results and proposed mechanisms as far as I can tell.

Opus 5.5 on low is good if you are relatively sure a given task is straightforward and time spent gathering background information is purely wasted or possibly even detrimental. Opus 5.5 low seems great at doing the most obvious thing based on the prompt without thinking much about it.

For some tasks I genuinely feel Opus 5.5 does better on low than medium or high. The trouble, as always, is figuring out which.

Edit: I realize after commenting that you're not suggesting medium is useless, but just arbitrary and differs between models. Personally I don't want to directly compare mediums between models, I just want to be able to compare across effort levels.

The absolute values aren't meaningful per-se, it's the relative growth that's mind-boggling.

I mean the process looks very similar even if the physical motions are different. That isn't the point, and honestly, I know you know that.

IIRC, it's based on the distance from the blast radius. The confounding thing about it is the fact that there is apparently a range of distance from the explosion where people have lower incidence of cancer than people who were even further way.

That seems hard to explain.

Passkeys are still a trap even if consumer passkey implementations don't currently support attestation. They are still ridiculously anti-consumer, overrated, and definitely still a risk for the open web because the door to attestation is right there.

Why does my bank (Chase) allow iCloud but not KeePass XC? Imagine if my bank got to choose what password managers I was allowed to use to write my password into. That is literally Passkeys, with optional cryptographic enforcement that people promise won't be used for anti-consumer reasons. (People. Not Microsoft or Google or Apple, but people.)

Secure Boot and TPM were also billed somewhat similarly in different times and now Windows 11 requires it and a lot of software and games require remote attestation with secure boot.

This cover story has already been used before, we already saw where it leads. Will people be foolish enough to believe it wouldn't happen twice?

TIL. That's good to know, makes me feel a bit better. I'm sure my dentist will want an Xray anyway next time I come in.

It's fiercely debated but as time goes on it seems like it is at least partially true. I'm guessing that some effects of radiation really do map better to LNT whereas others map better to radiation hormesis; why and when is a mystery, I think. Maybe some folks more vested in this topic have an idea.

The most fascinating example of hormesis I can think of is that after the atomic bombing of Hiroshima, some people who would've received a relatively small dose of radiation seemed to have a slightly lower cancer risk than people who wouldn't have received any additional radiation.

https://pmc.ncbi.nlm.nih.gov/articles/PMC6937943/

I'm sure there is plenty of debate about these studies and their methodology, but I have some suspicions. There seems to have been a lot of handwaving and intentional omission to explain away or ignore why LNT didn't seem to match reality. Radiation hormesis may be an illusion itself, but as it stands it appears LNT is unable to explain experimental evidence sufficiently. We're clearly missing something, and there is a lot of potential conflict of interest, especially given the obvious incentives for the energy sector...

But also, I just read Wikipedia and watch YouTube causally, so hell if I know.

Unnecessary medical treatments can have serious consequences. Even if you are absolutely certain you have a given condition, some treatments are very not worth it. More importantly, any given scan misses tons of important potential health issues. Not just the fact that CT scans don't show signs of MS vs MRIs, but neither of them will tell you about many cardiovascular conditions. So time spent treating made up issues from CT scans you don't need may be better spent doing other regular exams instead. Meanwhile, many aggressive cancers will be far too fast for any reasonable scan interval to catch in time. (And CT scans, to my understanding, can miss tumors early on, since it can't see as fine of detail as MRIs.)

I'm sure this has been discussed elsewhere, but I was recently befuddled at just how insanely the OpenRouter rankings graph has grown this year. Look at this shit:

https://openrouter.ai/rankings

Am I to understand the weekly pace has gone from 3.2T tokens to 181T tokens since last October?

AI will have many problems but it seems profitability (maybe) ain't one.

You can --yolo if your security boundary is a serious one, IMO. Although it's really only actually scary if you're running some absurd 1000 node swarm.

With modern X ray imaging devices I feel like the dangers of irradiation from dental imaging are probably quite low. If nuclear hormesis holds up, you may actually be better off getting the imaging done.

The problem is the same as the CT scan: as much as my health anxiety would love another set of CT scans to feel relieved about, even if they do find something potentially interesting it may be worse to act on it than to do nothing, while leaving other things that can't show up on imaging unnoticed, potentially diverting attention and giving you a false sense of security.

YouTuber and doctor Dr. Bernard "Chubbyemu" Hsu recently covered an upsetting example to demonstrate how doing additional routine scans wouldn't have helped and may have harmed in a specific sort of scenario that I'm sure many of us fear from time to time. It doesn't prove or even assert that such scans would be useless, only that they are far from a panacea and that it isn't settled science that we'd be better off with it.

https://youtu.be/YDYMMgyN4k0

And to me, I think that ties into how we're doing dentistry wrong: I think taking routine X rays for patients with healthy mouths, no history of issues and no symptoms is reckless. Maybe it is sometimes beneficial, but every single year as a requirement seems insane. Why not every 3? Are we really catching things early or are we drilling into teeth that are fine? I question it.

Of course, I'm just some chuckle fuck on the Internet with zero dentistry experience, but my knee-jerk is that something seems off here.