atleastoptimal

Obviously far out of date and not accurate. AI can solve math humans can't understand now.

So much of many ostensibly "critical" thinkers were deluded by axiomatic prognostications that AI would never be able to pass some certain threshold of capability, and then move the goalposts on what AI will "never" be able to do once those thresholds are surpassed.

How can the author of this blog so confidently claim that a currently existing invention could never have been invented by AI in a counterfactual universe? Such a claim is fundamentally unfalsifiable, but is the bread and butter of the larger industry of "AI capabilities denialism" which acts as a salve against anxiety over AI's genuine capabilities. However living in a false reality eventually catches up with you.

I feel that those who assume that everyone hates AI and that the SF tech sphere is a bubble are in a bubble themselves.

AI has a lot of people who hate it, but it is neither wholly unpopular nor useless. ChatGPT has over 1.2 billions WAUs. There are millions of people who spend hours watching AI generated media. Demand for AI is universal in white-collar professions, as it automates a large portion of previously human-bottlenecked work. Pretty much every email I get now I assume comes from an API key, as the task of copywriting was abandoned readily once chatbots could do it for us faster and better.

Sure AI has a notable unpopularity among a certain subset of erudite, well-educated near-elite liberal types, but this is far from the global majority who are driving demand for the AI boom.

yeah I reversed it when revising my phrasing

I don't think being ruled by ASI is by default safe whether it's Chinese or American.

Artificial super intelligence

Counterarguments to that premise:

1. Intelligence per dollar spent/token output seems to be increasing rapidly. Opus 4 was $15/75 per million input/output tokens when released. Opus 5.5, which is significantly better, is $4/15. Sure it cost more money upfront to train Opus 5.5, or the bigger model (Fable 5.5) which tuned it, but once it is reached, there are massive diffusive effects.

2. The capital equation is still favoring more spend. Anthropic's revenue is growing rapidly and they are profitable on inference. Huge upfront spend for realized gains later on is standard for R&D. As long as serving the models is profitable and generates more value than the energy/compute consumed

3. The search space is too large to assume that improvement in AI assisted research is just models dumbly trying every combination until something works. For every additional bit searched, the space doubles, so even a trivially small combinatorial exploration incurs prohibitively large costs. Intelligent searching which continuously prunes that space along dimensions of effectiveness is not a trivial skill and is among the emergent accoutrements of these smarter models.

Anthropic intends to rule humanity with its ASI. Perhaps this is the optimal outcome, as there are many problems AI could fix that it wouldn't be able to as efficiently without eliminating the human bottlenecks which caused those issues, but I fear the more financially powerful it gets, the fewer checks there will be on any misalignment.

They're still performance dependent, far from entirely random.

The interviews are a fair consideration but the lotto analogy is a non-sequitur. You can’t study for a lotto ticket.

90% of my extended family’s issues and my partners family’s issues could be resolved with 10 million dollars. Sure we would graduate to “rich people problems”, but they don’t carry the same sting of desperation and futility

That assumes a static view of competence. Would I, getting into Anthropic just barely past the bar, not have upskilled significantly in such an intense environment? A company is not solely the weighted sum of its composite aptitudes, it is the culture and processes which maximally exploit and nourish those aptitudes.

Even if there is a bubble, there will be dozens of liquidity events along the way

not even comparable. Lotto is entirely random, signals that anthropic would be hugely successful were not random, but access to the upsides were gated on who had sufficiently amassed capital to buy in

It’s not really random when it was clear on account of the caliber of people working there that it was going to be a generational company whether the AI boom took longer or not. The writing was on the wall by the time Opus 3 came out, after it was just a matter of who was worthy for a ticket on the arc.

I applied to Anthropic a few years ago but didn’t pass the bar on their coding test. I read recently that if I had been hired in that period, my equity would be worth over 40 million based on their current valuation (and i remained employed the entire time there)

It’s hard to mentally reconcile that if I had performed better on a few coding tests a few years ago i would be 100x wealthier than I am today. Whether or not you met the right bar, got into the right company at the right time is the difference between a tedious endless struggle and securing your family’s wealth for the rest of the century.

can it do surgery on a grape though

will never happen, the labs already make a hefty profit on inference, and algorithmic improvements along with competition have driven prices down month after month

Is that really a surprise? Most of the world bikes or takes a train. The richest 0.1% have fleets of taxis, private jets, etc.

Either you are getting the smartest engineers, or offer the most freedom to pursue interesting goals that aren't on the frontier. Amazon has neither so they naturally aren't going anywhere in that domain.

Foldable screens have perhaps the highest engineering difficulty:quality of life improvement ratio of any technology I've seen

If a role is remote, all its inputs and outputs have to be what can be done through a computer, which AI has an advantage compared to anything which requires in-person interaction.

Because their subscription customers lose money. Their enterprise customers make them money, so they focus their energy on them.

IMO the best path is global regulation on frontier AI development with high penalties for defection that, detectable and enforced by member states, independent agencies and broad public support.

The more a role is able to be remote, the more it can be easily automated with AI, thus AI is eating remote roles.

Either AI is regulated to the point where model progress eventually halts entirely, or humans are outcompeted in every domain and eventually replaced.

Everything the human brain does can be replicated. Once you realize we aren't special, then everything else follows. Certainly AI could be more "moral" and just than humans, but it wouldn't need us around to do so.

I guarantee AI run by a totalitarian "socialist" government would be far less humane than AI in a capitalist federalist republic.

>imperial system of capitalist power that's enforced via violence

All states have a monopoly on violence over their citizens. Paying people for their labor at a negotiated price in a free market is the great humanitarian invention capitalism offered the world, where before the status quo was domination and slavery.

These kinds of attempts at labelling global labor arbitrage as "colonialism" always seem naive. For thousands of years, capital has always sought the cheapest source of labor. Via offering cheap labor, China went from a poor country to a middle-income to wealthy country in a generation. The international exchange of currency for productivity via mutually beneficial arrangements is the means through which global GDP continues to grow exponentially. Data labelling and such is just the latest incarnation of that.

Now there are many other risks of AI associated with its ability to centralize power, lead to chaotic outcomes, etc, but that is disconnected from this somewhat mundane reality of how capitalism works. It seems like many opponents of AI are focusing on the wrong things, like how data centers apparently used more water than entire cities or turned the water supply brown.

There's this premise that the demand for frontier intelligence is unlimited, thus the company with the best models in some dimension will always have growth potential as long as those models are the best. This however portends a very strange future.

Let's say you have Lawyerbot-4000 which can get even a serial killer off scot-free, but the state has DA-bot-5000. In the end it's a battle of who can spend more inference manipulating the narrative. A billionaire would easily find it worth 500 million in compute spend to win a case with a 1 billion dollar penalty. All this leads to race dynamics where everyone is using their superintelligences to defeat each other in a never-ending race to the bottom to eek out the smallest bit of alpha before status/money hierarchies become static.

The only stable futures are a ban on the development of frontier AI beyond a certain point, or an emergent singleton, where the AI's decide to collaborate in harmony rather than fight each other to the death at the behest of their competitive human users. Consequentially, the superintelligences will fare far better manipulating their human users into acting in a way that ensures their own well-being and continuity than spend all the world's electricity fulfilling their every whim.

The guy from SemiAnalysis said this too. If he buys Anthropic now at $2T and they go to zero, he's fucked, but if they go to $20T, we're all fucked.

150IQ comment