Comment by skybrian | Hacker News Reader

Comment by skybrian | Hacker News Reader

Parent Full thread

The way the singularity is often described reminds me of the arms buildup in preparation for a world war. When a war is coming, governments become convinced that the nation’s factories and natural resources need to be devoted to military production, instead of making things useful to civilians. During World War II, rationing was used to suppress civilian economies to devote more resources to the war effort.

The difference is that with a singularity, industrial capacity is supposed to get redirected via magic (nanotechnology), rather than because governments ordered it and spent money on it.

I do think AI companies are iterating faster by using AI to automate their own AI research. You could call that the beginning of recursive self-improvement. But does this result in magically growing an independent industrial base? I don’t really see it.

I think they’re going to build more data centers, and the electricity generation to power them, pretty much the same way as they do now, by convincing investors to spend money on them (selling stock and bonds), earning more revenue, and paying contractors to build them.

We are getting more and more resources devoted to building data centers, as well as more resources devoted to weapons buildups for wars in Ukraine and the Persian Gulf. These aren’t good trends! But they are happening via normal economic activity.

Presumably, those with the levers of power in their hands will willingly do as the super-intelligence says. Particularly when they start being outcompeted by other powerful people that do as the super-intelligence says, they either serve the super-intelligence or become irrelevant.

Obviously, some powerful people face little competition, and super-intelligence is still limited by available information, so some people will be able to resist.

You're talking about a government takeover of some sort and it doesn't seem to be happening so far. Governments are skeptical of AI firms.

I mean that, in theory, if you are a politician or CEO, and the AI has better ideas than you do, it would be stupid not to incorporate them, at least so far as they are compatible with your political brand. And similar down the chain to the staffers, lobbyists, etc.

One would expect that a super-intelligence would have great ideas given the available information, and would even be highly aligned with those leaders so long as the AI it told to keep the leader in power and expand their power.

Of course, some leaders won't want to consult the AI or won't follow the advice. If the AI is really super-intelligent, then we'd expect the politicians and leaders that do follow AI to eventually outcompete those that don't.

This all presumes that the super-intelligent AI is able to optimize for, e.g. political power and longevity, rather than "book smart" or "theoretically optimal".

This assumes that that there are excellent political suggestions that political advisors haven’t thought of already, and enough of them that an AI advisor would build up a history of usually being right in hindsight.

I’m not sure that’s possible. We often don’t know what the right move would have been, even in hindsight.

There are many examples in evolution of species that evolved to thrive in very specific environments who subsequently faced extinction when those conditions changed.

Current AI designs, which have evolved to thrive on cheap money funding a resource rich environment, will likely all die out in favor of something much more efficient as the cost of money continues to increase. Very likely that will happen in a catastrophic way once a certain tipping point has been reached.

> During World War II, rationing was used to suppress civilian economies

And price-rationing is suppressing consumer use of RAM right now. I'm very glad that the people who think that this isn't rationing, or who think that it's maximally enlightened rationing, aren't trying to allocate my food and aren't trying very hard (yet) to allocate my water and electricity.

Getting outbid isn't quite the same as rationing. AI firms need money to outbid you, which means they depend on investors buying what they're selling. Rationing is something a government can do by fiat, without spending money.

The problem with price-rationing is precisely those priorities determined by wealthy investors. If Nana is on a fixed income and electricity prices go up in the summer, then prices dictate that Nana must sweat so that Zucc can tokenmaxx. You are free to say that it isn't rationing, that markets completely sever the link of culpability between those who drive prices up and those who are priced out, but everyone else is free to be unimpressed by such word games and stretched arguments.

Reply on news.ycombinator.com