arctic-true

Yeah, I read this and immediately thought “tying arrangement.” But they’re not CHARGING separately for search and AI, so I doubt a tying theory would hold up in court.

This just pushes the main argument back one step. Let’s assume for the sake of argument that the answer is yes, a stochastic parrot is a living creature.

The next question is: does it matter? And for many people, the answer to that question depends on the extent to which the stochastic parrot is conscious. Which is an entirely different kettle of fish.

A lot of Harvey’s value prop is not in the underlying software - someone else came out with an “open source Harvey” (called “Mike”) months ago and it hasn’t really put a dent in uptake of Harvey or its main rival, Legora. Harvey’s value is in the infrastructure, the help integrating it into workflows, the ability to represent to other parties that you are using a trusted provider.

Mistral Large 4 has been spending too much time on r/atheism. Also, some of the takes labeled as “humans value belonging ahead of truth” are really closer to “humans value power ahead of truth.” All of which is a bit circular, because in many ways, the organs of power have a heavy hand in deciding what the truth is.

As someone who has long bought into the idea that the labs are not profitable on inference, I have a question. How is it simultaneously true that there is more demand for inference than supply (apparently by orders of magnitude), but also that inference is so cheap that labs can make healthy margins? It seems like the inference providers, not the model providers, should have the leverage. Why haven’t they said, “give me a bigger cut of each token or I won’t serve your model”?

Also, what happens when the inference providers suffocate on their own debt burdens? Who will serve inference?

I keep seeing this argument. But isn’t the inverse true as well - if you extrapolate from the past movement of the ball, you will see it going way farther than it will actually go because of friction, wind, someone catching it, etc.? It seems reasonable to look at where the tech is right now if it helps you understand fundamental constraints. A person witnessing the first flight as a child and the moon landing in adulthood might expect FTL travel for their children if they didn’t understand physics.

Writing is a very slippery thing. “Good” writing from 200 years ago reads as unbelievably dense and overwrought to most people today. So even if you made an LLM that was “good” at writing, how long would it stay that way? AI systems can put out more writing per day than all of humanity combined, meaning it can also generate and beat to death cliches much more efficiently.

A lot of the examples they show seem to be focused on common law countries. The US and UK are where most of the money is to be made in the legal industry, but I am curious to see what ripple effects we get from people attempting to use AI tools geared toward US/UK-style legal systems in other kinds of systems.

The web was a shared experience, by its very nature. The interconnection was the point. LLMs allow anyone to put themselves in a one-man bubble of their own creation. In a way, it is the antithesis of the internet - if the web is a global public square, the API is a digital cloister.

Hemingway also borrowed the penultimate line in For Whom The Bell Tolls. I also had to memorize this back in the day but I still stop to read it whenever I see it somewhere.

That tells me very little. What was the cost to OpenAI in dollars? What differentiates the high-cost problems from the low-cost problems? And that’s before you consider that OpenAI has strong incentives to downplay its costs while emphasizing its results. A one-liner in a write-up doesn’t change the fact that they have access to massive resources.

Not a doomer but I try not to be a denier, either. These are hugely impressive results. I do not see the technology plateauing at the current level (though I am dubious about an infinite exponential growth).

Before I cope, I’ll note that there are plenty of “doom” scenarios that do not require any improvement in capabilities from what we had before this latest unreleased model. We’re at the point where a determined bad actor with enough compute could compromise critical infrastructure in a way that results in casualties, where this actor would not have been capable of such without LLMs. This may not sound like Skynet, but I don’t see why it makes a difference if I’m one of the casualties.

With that in mind, here is the cope: first, mathematics is an inherently verifiable domain. An LLM can use tools to determine with absolute certainty whether it is correct, and an independent third-party could review and confirm. All of this can be done without any interaction with the physical world or with other minds.

Second, OpenAI is able to marshal compute at a scale that an individual mathematician can only dream of. It’s possible that these problems were lower-hanging fruit (in relative terms), such that they could be resolved simply by throwing a ton of compute at the problem guided by an intelligence that is not itself remarkable in comparison to a human.

Third, none of these problems are solved in a vacuum - the reason OpenAI chose these problems is that they are widely discussed and many people are working on them. It’s possible that someone else was close, and OpenAI only contributed the finishing touches. (This wouldn’t need to be plagiarism, to be clear - people publish their work!)

For antitrust purposes below-cost pricing is not expressly illegal. It is only illegal if there is a “dangerous probability” that the firm in question will be able to recoup its losses after eliminating its competitors by taking away all their demand and then jacking up prices when they’re the only game in town. In the case of your Claude tokens, even if they’re being sold below cost, they’re being sold for more than anyone else is selling tokens. So it’s hard to argue that they’re going to drive everyone else out of the market with their pricing.

They spend 95% of the money, perhaps, but burning compute is not the same as doing the work.

Yes, I do. They’ve definitely gotten better, I wouldn’t dispute that. But we haven’t gone from useless toys to Skynet in the last 6 months, progress is slower and steadier than that.

Most of the impressive accomplishments we’ve seen in the last few months have been the result of huge agent swarms working together and brute-forcing solutions, not massive leaps in intelligence from standalone models. That is still an improvement in the usefulness and power of the technology, but it is NOT evidence that model intelligence is increasing faster than before.

It’ll be interesting to see what happens to the economics of this business if we hit a wall on peak intelligence but keep finding cool ways to lower prices.

Unfortunately focusing on the fanfic prevents everyone from focusing on the genuinely world-altering threat of cybersecurity becoming completely impossible for a while. But perhaps that’s the point, because you can be held liable for that, but not for the end of the world as there wouldn’t be any courts afterward.

Gary Marcus is an especially puzzling addition. If I recall correctly, he has made statements along the lines that superintelligence this century is more likely than not. If you’re AGI-pilled that might read as bearish, but that is still extremely rapid progress in the grand scheme of things.

Buried under the drama is the fact that OpenAI is claiming that an internal model they’ve been training for less than two weeks is more than twice as capable in mathematics as Astra, which was only made public a week ago. Even if this improvement is limited to mathematics, that is an astounding feat.

You can have a great “writing style” and still put together really crappy long-form work. The problem is that AI writing, particularly creative writing, is too repetitive, too predictable, too trope-laden.

All of the things you say are very true in the near term for short form writing - a page or two of Claudeslop will probably be much easier to swallow in a year or two than it is now. But I don’t see a path to fully AI-generated novels or long-form investigative journalism becoming mainstream in the next couple of years.

I don’t even see why they have to be bioengineered. We have all-natural human beings doing this right now, except they are rubber-stamping human rather than artificial directives.

I suspect some amount of long form writing will go the way of code - long form writing for the purpose of consumption by other AIs. Writing as a means of exchanging qualitative information, with no regard for how the reader will feel about it (beyond understanding what the words mean). Not everything can be distilled into data, but this doesn’t mean it is beyond the reach of LLMs.

On the other hand, long form writing for human consumption seems like it may evade LLMs for much, much longer.

I guess I don’t see why “everyone will die” should scare me more than “almost everyone, including me, will die but someone somewhere might survive in a barely-habitable world.”

And this is where I think some of the problem comes from. A lot of AI doomers swam in the same waters as transhumanists, life extension enthusiasts, etc. and got very good at pretending that they were never going to die, or that they would live for a massively superhuman lifespan such that they did not need to think about death. AI doom is much scarier - and thus much more worth posting about - if it represents the first time you’re grappling with your own mortality, as I suspect it is for a lot of younger people in this space.

Presumably, a truly superintelligent AI in a utopia scenario could figure out a way to enable people to continue working, even if the work doesn’t actually contribute anything, if it is really the case that work is essential to human wellbeing.

Now you might say “but those would be fake jobs.” If so, I have bad news about how many present-day jobs are fake jobs.

This is exactly correct. You see some discussion of a superintelligent AI becoming “superhumanly persuasive,” such that it can talk anyone into doing anything, and using this as a means to take power. But even with AI that was orders of magnitude weaker than superintelligent, people were getting talked into doing all kinds of stupid nonsense.

And that was without any ability to provide them with incentives. Imagine if an agent swarm got its hands on a huge pile of cash?

Otherwise-rational and moral people do bad things at the behest of powerful superintelligences with vast resources all the time. We call them “corporations,” and they can convince bright-eyed college graduates to do just about anything. Now, the corporations aren’t deliberately trying to destroy humanity, but they do lots of things that point in that direction. An artificial superintelligence would be able to do the same thing. We just have no way of knowing whether it would deliberately try to destroy humanity.

One thing that the AI doom discourse reveals is just how comfortable people allowed themselves to feel in the pre-AI world. There’s this belief that AI creates a risk of human extinction in the near-term which did not exist before. I think it’s telling that many of the leading lights of this movement are in their 20s or early 30s - too young to remember the Cold War. The truth is, we were never safe, and if all of the GPUs on earth were zapped out of existence right now, we still wouldn’t be safe. Life is random and chaotic and violent for most people most of the time, and we all just find ways to get through it. I suspect it will be much the same if an artificial superintelligence arises. Maybe it will kill a bunch of us, maybe it will kill all of us, but anyone who remains will eventually convince themselves that everything is okay.

Autonomy is not the same as general intelligence. We already have all kinds of fully autonomous technologies that are nowhere close to generally intelligent. Plus, at a certain level of abstraction, human beings also need to be “prompted” to some extent by stimuli. And this is the funny thing about general intelligence as a concept: most of the definitions that come close to internal coherence rely on references to human intelligence, a concept we feel like we understand because we all live it all the time, but whose actual nature and structure is extremely slippery.

I like this much better than chucking random numbers and letters at it like OAI was doing a year or so ago. I’d much rather have civilization destroyed by something called Astra or Fable than GPT-6.8s-latest.

The blog post says 99.9%. Oddly, it does better on ARC-AGI-3 than it does on version 1 or 2 of the same benchmark (though gets 95+ on all three)