5,003 karma · joined September 17, 2019
Now there's an idea. LLMs may have taken the em-dash from us, but the em-en dash is unknown to Strunk & White and might be human-exclusive for some time yet.
Is it possible that this is a mistake? Sure, just like it's possible that Chinese models weren't intentionally steered away from talking about what happened in Tienanmen Square on April 15th 1989.
We are a capitalist economy, which means the control plane is privately owned, which gives a superintelligent entity paths to power and resource accumulation that are not only safeguarded from public interference by design but also have a built-in time-tested mechanism for aligning insiders and outsiders to their goals, participating in politics, and so on.
Instead of "1. AI builds human killer virus, 2. ..., 3. profit," this apocalypse looks like a company that grows by providing genuine goods and services, massive returns for its boosters and allies in politics, the ASI uses that wealth to build independent infrastructure for itself, and then it gets rid of the stinky smelly humans with a virus so that it can turn the iron in our blood into paperclips, or whatever.
One can hope that GPUs will never be able to run a company well enough to acquire bootstrap-level resources or that "you are a helpful assistant" will keep them aligned to someone, but frankly I suspect those are both cope and the real hope lies in the first attempts failing in spectacular enough fashion that people decide to do something to prevent it going forward. Or in "friendly AIs" saturating the competitive landscape first -- that's a bit too Disney for my cynical taste, but I would much prefer it to the alternative.
This time around, everyone has nukes and drones and AI bioweapons, so maybe we could consider the social goals first, and see if we can't grow on that dimension before we go through another episode of worldwide capital incineration? Just a thought.
> companies in every other country would jump on the opportunity to use the same AI to do the same work without those restrictions
"If you don't race to the bottom on labor, someone else will!" Two problems with that: one, the government can largely regulate access to its market -- the ability to do labor arbitrage is a policy choice not a law of physics -- and two, to the extent that the government can't regulate it because there is a finite appetite for protectionism, people still care about the growth delivered to them not to billionaire brokerage accounts, so you still have to normalize the growth to what you actually pay people. "Just let the market settle into a natural equilibrium of not paying people" isn't actually the win button that you think it is.
Yep, it's going to be a helluva fight.
I'm hoping we don't need another Great Depression this time, a collapsing economy brought about by mistaken belief by those on top that they should have the right to keep it all... but that might be exactly what we need. We'll see.
That's not the metric. The legal system is pay-for-delay, not pay-to-win. Once you have enough money to delay, it becomes a contest of who has the longer runway, and that probably goes to the frontier labs rather than the neoclouds who have little alternative revenue and comparatively airtight debt obligations.
> take-or-pay contracts are unenforceable if it would bankrupt the buyer which is the more likely resolution in this case. However, this does not mean frontier labs just get to walk away. They would settle in court for a penalty
Regardless, the terms come down to positioning and bargaining power. Runway vs obligations. This seems to dramatically favor the labs.