Current and former AI employees warn of the technology's dangers | Hacker News Reader

Current and former AI employees warn of the technology's dangers

washingtonpost.com

26 pointsyaya_yazi52 comments

It’s pretty interesting how widely divergent views of the current state and promise this technology is. The dominant view here on HN seems to be that generative models are overhyped toys that burn massive amounts of energy and investment capital with little signs of promise. Meanwhile you have these AI researchers who are warning about “the loss of control of autonomous AI systems potentially resulting in human extinction.” What are we to make of this?

I think in most cases it's wrong to treat HN opinion as a monolith, including in this case. There certainly are a lot of people here who think that LLMs are overhyped, but there are also many who find the current tools useful and expect them to improve dramatically in the near future. I think most people have a more nuanced view though.

There is without a doubt a lot of marketing and hype bullshit around AI right now, but that doesn't mean that there aren't reasons to be optimistic about LLM capabilities.

I do think that the strong divergence of opinions among knowledgeable people is interesting. No matter what your position is on this topic, there are smart people arguing against it. The thing I'm most confident about is that anyone being overly confident is likely to be wrong.

> The thing I'm most confident about is that anyone being overly confident is likely to be wrong.

So does this mean you're wrong? This sentence really amused me.

No, his is a second-order statement that applies to first-order statements. It's not intended to apply to second-order statements.

Remember, "the loss of control of autonomous AI systems potentially resulting in human extinction" is just the very end of the spectrum of risks they're flagging. There are still other very serious risks experts are calling out. I hate for the less controversial risks to get drowned out by arguments over the most controversial ones.

Completely agree, but for that reason also think we need to be more specific when discussing AI safety. Both "AI" and "safety" are overbroad in the extreme.

The current systems don’t have the capability to update their knowledge and can’t really perform complex reasoning.

The obvious approach would be to attach a proper memory somewhere between layers and farm out reasoning to a sat solver, also preferably between layers. This sounds very sci-fi but chatting with a computer has been sci-fi 20 months ago and it’s a commodity now.

Some of the AI researchers are probably thinking about future systems only somewhat related to today's systems.

Couple of things come to mind. This is a community of people that:

- are 'in the room' when evidence of new capabilities emerge

- are 'in the room' when discussion of new use cases and applications are held

- have an intuition about the raw rate of advancement and how it changes over time

Imagine you're 'in the room' during the Manhattan project when someone says 'hey, there's a nonzero chance that a test will ignite the Earth's atmosphere'. Vanishingly small maybe, but of incalculable impact. These folks may be the sort that say something rather than not.

Unfortunately, we are at the point where people's identities are wrapped up in this.

I would love it if people were having well reasoned discussions of the risk of AI, that are grounded in actual evidence of what capabilities AI has today, and how they might incrementally change in the future to cause more or less damage.

But unfortunately whenever I try to discuss grounded risks, I get responses that reference the anthropogenic principle or other similar unfalsifiable claims. The arguments amount to "Well, of course I can't show you evidence of how AI is going to end the world! If I could, then the world would already be over!".

Which is an awfully convenient excuse for why your arguments are a special case that don't have to proved with evidence, like any other claim. Its perfect for a side that knows that it doesn't have any!

I can be convinced out of my position, with evidence. But I have no idea what proof I could possibly show the other side to do the same for them.

Per usual, it's somewhere in the middle. I find LLMs to be very useful for coding and solving quick tasks, but I don't think it's going to turn into skynet or make humans worthless.

There's a lot of momentum (driven by incentivization) around sensationalizing and storyboarding emerging technology into the "to the moon" and "neo Luddite" camps, and there's far too many people that fall into that uncritically.

My guess based on historical precedent of most emerging tech is that ten years from now AI (my preferred name is machine learning) will settle in as a useful tool for certain tasks, and we will be shrugging about the implications of it.

The cynicism (in my case, and probably others') comes from remembering self-driving cars, blockchain, crypto, the millennial internet hype, and previous AI hype. Smartphones have arguably made life worse for many people too, judging by skyrocketing mental illness.

Edit: oh, and Uberisation (workforce casualisation), and AirBNB. Enshittification all round.

They might be right this time, but the prior has to be this is yet another hyped nothingburger, or more enshittification.

Why does it have to be completely worthless or completely dangerous? Where does this binary thinking come from? In my experience, almost nothing in the world is binary on a spectrum of "good" or "bad".

“Millennial internet hype” was absolutely warranted, though. The utility of the Internet has exceeded the expectations almost everyone had for it in the 90s. It’s one of the big things people pushing blockchain hype always cited for their faith in the technology.

You will have a high batting average if you are dismissive of everything that gains momentum. But when you are wrong you end up being extremely wrong.

This is how I assume one of the ways an expert system would play out:

1. A system which understands not just the language but the meaning behind it and the cultural context around it.

2. Let's assume it's multi-model and have all the senses having the same level of technical maturity.

3. It has read all the text available for all the major sciences and understood the meaning behind it including the possible applications of it.

To that end, having the above conditions satisfied I am assuming there should be major breakthroughs in all sciences. So far what I have seen is that the AI system is able to selectively brute-force it's way in finding a solution. Maybe I am just being impatient.

They are saying what will offer the best hype and a good reason to be leaving.

The reality is that they have shown the world some pretty big, powerful offerings they are working on. That value is already priced into their stock.

If they fail to deliver on the hype, which they will, it will tank their value hard. If they meet the hype expectations (no one ever lives up to the hype, ever), there will be no jobs and no employees and no businesses and thus their stock will tank.

I believe that, more realistically the easy, low hanging fruit has already been plucked and they are leaving now before everyone realises the emperor has no clothes on.

The technology is indeed powerful; it's pretty obvious you can make military grade or better propaganda pretty easily with this technology and disseminate it more easily. Let's just classify that as "misinformation" generally, which could also be applied to ideas of teenagers making deepfakes of classmates, etc. So misinformation is a big risk, and already affecting society. It is also clearly useful to whomever is creating the content for whatever their means or ends may be.

There are definitely use-cases being worked out for this technology, and the people who are developing these are using this technology on a daily basis (or creating it); their proximity to the technology and gain in understanding of it probably is helping them realize how this could be truly subversive and an existential threat.

What is "military grade propaganda"?

https://www.usmcu.edu/Outreach/Marine-Corps-University-Press...

Propaganda is a crucial part of the military; there are even divisions in modern militaries whose sole job is create propaganda/misinformation. This material is clearly deemed of some quality or grade that would deem it possible effective against a target. Given that it is the job of these departments to influence the information and human psychological "playing fields", then I would give that context a label of "military grade" -- in that it is as powerful as information warfare techniques used by entities whose sole job it is to engage effectively in information warfare.

That's funny, I always thought military grade meant overpriced and underbuilt.

I mean, I would assume that they're not necessarily talking about LLMs.

And the 'oops, we made Roko's Basilisk' risk is not the only risk that people are concerned with, of course. LLMs come with many of their own risks, primarily around misuse (ie using them for anything important.)

AI researchers are just as prone as everyone else to be self-delusional.

Researchers are people too and incentivized to come to conclusions which results in continuing their employment

This is the letter: https://righttowarn.ai/

See submission here: https://news.ycombinator.com/item?id=40574351

Seems like a pretty transparent call for regulatory capture with no specifics.

- "OpenAI Insiders Warn of a ‘Reckless’ Race for Dominance " -- https://www.nytimes.com/2024/06/04/technology/openai-culture... -- https://news.ycombinator.com/item?id=40574355

- "OpenAI Employees Call for Protections to Speak Out on AI Risks " -- https://news.ycombinator.com/item?id=40576018

https://archive.ph/nnvgy

What are they going to do to harm us, and how are they going to do it? I'm seriously asking.

[dupe]

https://news.ycombinator.com/item?id=40576018

Isn’t taking risks just a standard part of technological innovation?

Hmm, this is a common line of thought I saw in other young men around me when growing up...

[Takes risk]: See I took a risk and nothing happened.

[Takes risk]: "Dies horribly and tragically in a completely avoidable fashion after taking excessive risk"

So is mitigating them

I'd like to build a chatbot arena with pro-humanity questions.

Periodic reminder: at the moment, governments worldwide are literally developing killer robots which can’t disobey their orders. Your fears are comically misplaced.

No, not misplaced at all, both need monitored and regulated. And large companies like Google and whoever else will gladly sell their model for use in the latest killbot for some of that unlimited military industrial complex money.

Point is, people on this very site are cheering on unmanned plane aerobatics while hyperventilating about linear algebra and associative memory which might at times tell you to eat rocks.

I didn't realize we all represent the exact same things. It's like that sometimes when you forget you're part of the hivemind.

Myself, I see a mix of people, some disagreeing with the MIC and others gung ho for the MIC.

A heat-seeking missile or nuclear bomb can't disobey orders either. Therefore we should not worry about the dangers of guns or knives.

Heat seeking missile or a nuclear bomb have to be launched by a human today. Not for much longer though.

I worry AI safety is going to be a huge political grift soon

Soon? Seems like we're already there.

When China & India form an alliance and send a peace keeping army to protect the former citizens of the fallen American Empire Megacorp, then the American public might have reason to have regrets about AI. But until then it’s just another distraction.

Lmao. As if chatbots are going to destroy the world. Give me a break. LLMs, while having some cool uses, are clearly just the next in line of a never ending tech hype scam cycle. Honestly I'm kind of dreading what's coming next because each iteration just gets more insufferable.

It's pretty obvious that LLMs can be used to give instructions to machines, those machines could be hooked up to other sensors, like cameras with facial recognition, and actuators. Now imagine a Boston Dynamics DOG robot with a machine gun that can target specific people. This is already possible. If someone were to manufacture and let loose say 100 or 1000 in public, that seems pretty drastic...or flying drones, etc.

LLMs cannot reason so having them control robots wouldn't work. They are reasonably good at text prediction. That's literally all they do.

You seem stuck on this idea of intelligence, when you don't need "reasoning" to do any of this. You don't even need the LLM. You could just use object recognition and current robotics techniques. The LLM could just be the language interface for giving it instructions of where to patrol, what to observe.

https://www.youtube.com/watch?v=Sq1QZB5baNw

yea, you're way late to the game.

In general you should be very skeptical of these types of videos. Most of them end up being misleadingly edited or almost outright faked. There's 3-4 cuts just in this 2 minute video alone.

When assessing if the company is worth investing in, assume it's faked.

When assessing threat models, assume it's real.

The particular problem that we have is too many robot companies are putting out videos of robots doing things they have not been able to do in the past. Moreso, even yourself, you can feed data into multimodal models and get instructions to perform real world actions.

All that said, with your post history I'll have to assume you assume everything is fake and I'd more luck explaining it to an LLM.

They might! Imagine if an LLM went full Vogon and attacked us with weapons grade awful poetry. We wouldn’t stand a chance!

Reply on news.ycombinator.com