staticshock

While I'm sold on the fact that modern nuclear can be built & administered safely in the face of natural disasters, and is a net good environmentally, I'm worried about corporate cronyism's corrosive effects on safety (a la Fukushima) and future instability in the form of cornered animals (e.g. Putin, Trump) acting erratically by bombing civilian infrastructure.

For the uninitiated: https://en.wikipedia.org/wiki/Torment_Nexus

Everything is search.

Software development is search through the space of useful/interesting automations. Business is search for product market fit (at the intersection of expertise, capital, problem, etc.) Writing is search for lossless, efficient idea transfer.

AI software development is more search. If we search more, will we find a bunch of garbage? Hell yes. We'll find a TON of garbage. That's not new, though. The world has been writing way more books than you'll ever read, recording more music you'll ever hear, filming more television shows than you'll ever get to watch, etc. A lot of it is garbage, but the good stuff stands the test of time and rises to the top, and I'd rather live in a thriving, flourishing world full of all these things, because there's more cream-of-the-crop when there's more everything.

It's evolutionary fitness operating in the space of ideas. I agree that "maybe later" was indeed a useful mechanism, and maybe even a local optimum in the development methodology search space (which recently experienced a major earthquake!), but evolutionary pressure will bring it back into existence in some form sooner or later.

An agent you can sue would count as a legitimate third party entity. Suing today's agents won't get you very far.

Feels like an unfair judgment of someone else's values, which have no obligation to match yours.

Are you saying that a better way to measure perceived time is something like "1 year, 2 more years, 4 more years, 8 more years," starting from birth, and maybe call each of those increments a "log year"? I like it.

I guess the "natural" base to use to get the "right" number of increments is a pointless exercise, since it ultimately bottoms out in the question of "why is a regular year as long as it is?", but if we assume a base of 2, I'm currently in my 6th log year, and hope to die comfortably into my 7th. Actuarial odds are >80% in my favor.

Love it!

Bug report, if you're the author: I can't delete periods.

Feature request: I wanna be able to move periods from one layer to another.

"can be" ≠ "must be"

A pattern I like for CLIs is that by default each command runs in dry-run mode, and only with `--commit` is it allowed to do dangerous things. Kind of like `git clean` vs `git clean --force`, except that `--force` feels like bad names for the distinction. Likewise, `--dry-run` implies that the command does the dangerous thing by default, which is bad. `--commit` gets the balance right, it sounds right, and it's sufficiently self-explanatory.

(Oh, and there's no shorthand, like `-c`. It's `--commit` or bust.)

That song is etched into my memory and bring back a flood of great feelings. Daily Motion is giving me a playback error, but here's a copy from youtube: https://www.youtube.com/watch?v=xb_PyKuI7II

Contrarian thinking can be great because it taps into the intuition that the masses are mostly followers who can be led anywhere, not critical thinkers who've deeply examined what they believe. Being contrarian, then, is akin to staking out a new leadership position.

The space of contrarian ideas is vast, and most of them are probably bad, but, nevertheless, the willingness to hold unconventional, internally consistent views should be celebrated, because it increases diversity of thought. Our collective hive mind grows stronger through heresy.

However, I like my heresy with a splash of axiomatic precision, which is sadly lacking in this article.

The value of such a benchmark, to me, would be, "what is peak performance", not just "what is mid-tier performance". Also, possibly, "what's the per-dollar performance". Time and money permitting, I'd really want to see your benchmark extended to the large reasoning models.

AGI is 100% possible, even if the current breed of transformer-based models are not it, and even if silicon is not it. There's nothing special about human brains that we won't eventually be able to match (and then exceed) in vitro. We are living proof that intelligence can be built out of matter, and that human-scale intelligence can run on 20 watts. It's not a matter of if, but when.

Feels to me like idealism crossing into realism. OpenAI could be the next Google, or the next Facebook, or the next… I don't know, Netflix?

All those companies (and many other large tech companies) have discovered the same arbitrage that older media companies discovered decades ago, which is that we, on the average, are much more willing to pay with attention than with money, even where money would have been the better choice.

Advertising continues to be one of the most powerful business models ever invented, and I don't think that's changing any time soon.

I just looked up "may you live in interesting times" and learned that it is not, in fact, originally Chinese. Per wikipedia:

> The nearest related Chinese expression translates as "Better to be a dog in times of tranquility than a human in times of chaos."

https://en.wikipedia.org/wiki/May_you_live_in_interesting_ti...

> What we really need is to recreate software from a subjective perspective.

What does "subjective" mean here? Are you talking about just-in-time software? That is, software that users get mold on the fly?

Yep, there are tons. Growing food, building shelter, etc. But, for pretty much all of the skills we've allowed to atrophy in response to the advances of capitalism, technological & scientific progress, and societal changes, one COULD make the same basic argument, which is that losing that skill is detrimental to the individual, and yet here we are, not growing our own food, not building our own shelter, etc.

The arguments you make ≤ the values you actually hold ≤ the actions you take in support of those values.

I'm only interested in any such argument to the extent to which you've personally put it into practice. Otherwise, you're living proof of the argument's weakness. (To be fair, it's extremely hard to be internally consistent on this stuff! We all want better for ourselves than we have time and energy for. But that's my point: your fully subconscious emotional calculus will often undercut at least some of your loftier aspirations. Skills that don't matter anymore invariably atrophy due to the opportunity cost of keeping them honed.)

Hah, right, I mixed it up!

That's true, but I think it's beside the point. The flip side of that argument, which is equally true, goes something like, "not doing cognitive push-ups leads to cognitive atrophy."

There are skills we're losing that are probably ok to lose (e.g. spacial memory & reasoning vs GPS, mental arithmetic vs calculators), primarily because those are well bounded domains, so we understand the nature of the codependency we're signing up for. AI is an amorphous and still growing domain. It is not a specific rung in the abstraction hierarchy; it is every rung simultaneously, but at different fidelity levels.

The eloquence with which this point gets (repeatedly) made is continuing to improve each next time I read it. However, I still feel like we haven't nailed it. That is, we are not yet at the "aphorism" stage of the discourse (e.g. "the medium is the message", "you ship your org chart", "9 mothers can't make a baby in a month"), in which the most pointed version of this critique packs a punch in just a few words that resonate with the majority of people. That kind of epistemological chiseling takes years, if not decades. And AI certainly won't do it for us, because we don't know how to RL meaning-making.

Edit: 9 babies → 9 mothers

I can't believe this article does not mention what I think is the most puzzling part of the repair: the delicate process by which the individual fibers are FUSED TOGETHER in a way that maintains near perfect total internal refraction.

Not currently, but I have faith in our collective ability to push in the direction of such systems over the long arc of history.

In a well functioning system, the incentive to make money is somewhat aligned with the incentive to create value for other people.

I do the same. Gmail gives me a single, standardized interface for opting out of emails: mark it as spam. All the various companies I've given my email to, on the other hand, give me different, either clunky or often outright broken interfaces for opting out. There's no direct financial incentive for them to invest in making ethical, robust opt-out systems.

However well meaning, collectively all those companies are still just a bunch of sociopaths. This might be a bit dark, but I think a reasonable real world analogy here is stalkers and restraining orders. A stalker isn't motivated to listen to you when you tell them to stop talking to you. That's why you get the restraining order.

They were taught not to read errors because they encountered thousands of errors (in other software) that were less helpful than that one.

Most people have an adversarial relationship with software: it is just the pile of broken glass they have to crawl through on the way to getting their task done. This understanding is reinforced and becomes more entrenched with each next paper cut.

You know the saying, "when you owe the bank a million dollars, that's your problem, but when you owe the bank a billion dollars, that's the bank's problem"?

I suspect the theory behind OpenAI is to grow to be "too big to fail" as fast as they can, because once they cross that threshold, their liquidity/solvency problems will cease to be theirs, and become everyone else's.

LLMs seem to me closer to Kahneman's System 1 than to System 2. When understood in this way, it is obvious why LLMs are bad at counting r's in "strawberries". But it also makes ZEH feel like it couldn't possibly be a useful metric, because it's a System 2 evaluation applied to a System 1 system.

here's another good one: https://terraformindustries.com/

> the death of Payton Isabella Leutner

she's alive, so "attempted murder" would be more appropriate.

i really enjoyed the "we are 3D printers for our thoughts" framing!

I think the way to see this as the organic process of discovering hard-to-game benchmarks. The loop is:

1. People discover things LLMs can kind of do, but very poorly.

2. Frontier labs sample these discoveries and incorporate them into benchmarks to monitor internally.

3. Next generation model improves on said benchmarks, and the improvements generalize to improvements on loosely correlated real world tasks.