Comment by nonrandomstring | Hacker News Reader

Comment by nonrandomstring | Hacker News Reader

nonrandomstring

Normally I finish Bruce's essays with a sense of clarity and having read a sharp analysis. Not saying there's anything technically amis with this one, it's all terrifyingly clear, but I think he bit off a little more that he could chew. The result is indigestible. I personally would have split this into several shorter pieces.

When it's all put together this way though, I can't help think we've shot ourselves in the foot. As computer scientists and developers we've probably already lost control.

We owe a debt of honesty to the world to say so now and stop pretending otherwise. Then we can, as a society, revisit Postman's Seven Questions:

1. What is the problem that this new technology solves?

2. Whose problem is it?

3. What new problems do we create by solving this problem?

4. Which people and institutions will be most impacted by a technological solution?

5. What changes in language occur as the result of technological change?

6. Which shifts in economic and political power might result when this technology is adopted?

7. What alternative (and unintended) uses might be made of this technology?

The problem is that any answers that anyone comes up with for new technologies are completely worthless. No one knows how any technology will play out or its consequences. Read about the confident predictions people made for air power once the airplane was invented, and compare that with the reality: it's not that air power made no difference to warfare, but it made a much smaller difference than people imagined when it was first invented.

Secondarily, the people most eager to answer such questions are the people like Postman, people who are Concerned as a hobby, the people most eager to tell us that we are "Doing it wrong", and only if we listened to them we would be happy.

Do you have some sources where I can read about the wrong predictions of air power? It is a bit hard to imagine predictions overstating the importance of air power in relation to their actual usefulness in modern warfare.

This is a good primer:

https://acoup.blog/2022/10/21/collections-strategic-airpower...

The parallel with AI GP is making is very interesting. People at the time thought air power would be transformative in ways that didn't pan out, while missing more subtle applications.

AGI isn't comparable with any prior technology because we have spent the last ~million years being the most intelligent thing on this planet, and AGI seeks to upset that.

If you have to make a comparison, the most apt would be the introduction of invasive species to existing ecosystems. Any one existing species may be OK after the introduction of an invasive species to its ecosystem, but it usually ends badly and introducing invasive species is generally something we try to avoid doing to other species.

Of course AGI will not compete with us for food, but it will completely destroy our present economic system by creating vast unemployment.

Conditions of severe economic change with an unemployed population and huge amount of technological power in the hands of a few, seems ripe for producing totalitarianism.

I have nothing to reply except that as far as I'm concerned this is fanfiction.

It's a potential future. Nobody knows what will happen post-AGI. But that doesn't mean we shouldn't think about it.

I contend that people who think that it's going to be fine are probably extrapolating from their own recent past (i.e. their life has been relatively comfortable so far), rather than humanity's past, which is quite a bit darker.

> Do you have some sources where I can read about the wrong predictions of air power?

The Bombing War: Europe 1939-1945 by Richard Overy is a great and comprehensive start.

Very approximately, air power appeared in WW1 where it was essentially tactical. In the UK, the Army's Royal Flying Corps was then re-established as a post-war separate, independent entity (the Royal Air Force) and in the intervening period convinced many that a strategic bomber force could defeat Germany largely on its own. This was supported on a general hysteria about the apocalyptic power of the bomber (itself based on early SF by people like HG Wells, in which cities and morale would be devastated by fleets of bombers) as well as the imperial experience, where the RAF was used to police provinces from the air (e.g. Somaliland and Afghanistan).

However, when WW2 broke out, the effects of strategic bombing were initially negligible, with massive aircraft / crew losses for minimal strategic effect, and it took years for the bomber force to achieve the weight necessary to disrupt German manufacturing.

> answers that anyone comes up with for new technologies are completely worthless. No one knows how any technology will play out or its consequences.

The "Experts. What do they know, huh?!" thing is a little dishonest. Wouldn't you rather give Einstein the benefit of the doubt; that someone who is smart enough to design an atomic bomb is maybe smart enough to understand what it will do?

> the people most eager to answer such questions are the people like Postman, people who are Concerned as a hobby

No. It's not like these are different groups of people, one virtuous and pure in their desire to "make the world a better place" and another band of unwashed neo-Luddite primitives who jeer and throw rocks. What you're doing is called "splitting" [1]. We desperately need more people like Postman (even better if they have CS degrees too) to relieve us of the delinquent fantasy that "no consequences can be seen".

[1] https://en.wikipedia.org/wiki/Splitting_(psychology)

Thanks, I had not seen these Seven Questions before. As a pragmatist (?), I suspect answering from 3) on down is near impossible* unless you actually release it into the world. People are clever in a way I don't think any of us can anticipate and put to use things in novel ways that often surprise their creators.

* Well, at best it would devolve into arguing back and forth and neither side able to convince the other what problems would or would not be caused or are avoidable.

> As a pragmatist (?), I suspect answering from 3) on down is near impossible* unless you actually release it into the world. People are clever in a way I don't think any of us can anticipate and put to use things in novel ways that often surprise their creators.

That is likely true in many cases, but, even if definitive answers cannot be provided, the questions should still be asked in advance and the answers pondered. Many people take that difficulty or impossibility of conclusive answers as justification for not asking those later questions.

Well, you can't "idiot proof" most things, because the "idiot"s are so ingenious.

In this case though, I think it'll be the big guys who figure out how to fill the world with cheap almost humanly creative garbage. Whether it's spam, social engineering, or attention addiction they'll monetize it first and keep the little guys in mostly walled gardens of "creativity".

I feel HN has regarded blockchain and smart contracts with total derision as a solution in search of a problem. And yet with AI, it can actually harm the world but since it does impressive things, these questions are barely asked.

Progressives have started to talk about how “tech bros” think technology can improve everything but it also creates massive problems they IGNORE.

Blockchain has harmed the world. Ransomware, numerous scams, greenhouse gases... Before all of those harms, very few people had much to say about Blockchain outside of utopian visions.

AI has yet to cause harm. If/when it does, I'm sure the conversation will, likewise, turn sour.

It will be too late. These bots can be easily proliferated and attack in swarms, to infiltrate any forum shout down any group they don’t like, sow dissent, astroturf revolutions. People keep investigating how one bot will fare among a poker table of 8 players. No one seems to consider that these bots can very cheaply SURROUND all the humans 99999 to 1.

It is the nature of technology that we rarely grasp how it will change things beforehand.

But how novel is that really? Spam is already an issue. You solve it at the account level, not the message level. Trust graph forever.

Spam doesn’t solve the Turing test interactively and fool millions of your fellow users at scale.

Reply on news.ycombinator.com