Comment by zzzeek | Hacker News Reader

Comment by zzzeek | Hacker News Reader

it's so frustrating how there are really two conversations going on about this.

one is, "AI will soon enough be AGI and then the extinction of humanity is at stake"

the other is sort of paraphrased in this article: "These models are being unleashed into the wild by corporations who offer no transparency as to their corpus, their architecture, their guardrails, or the policies for handling data from users. "

one refers to harm that would be caused by some hypothetical intelligence that so far only exists in science fiction, the other refers to harm being committed right now by humans who own companies.

The various "letters" being signed seem to be firmly in the "AGI human extinction" category and make no effort to suggest that perhaps the training sets being used for LLM should be *opened up right now and limited to those sources that provide consent only*, that is, stop stealing art, music, writing, source code, photographs, literature, and sticking it in your for-profit product. Skynet is not doing that, Sam Altman is.

It's understandable: opening the training sets would increase the risk of the hypothetical happening, so it's difficult to expect the first camp to advocate doing it.

As for "stealing", it's controversial if "stealing" is even happening, given the plurality of laws and opinions, so it's an even thornier subject to pick up if you have other things to worry about.

Let those in either camp fight for what they consider important.

I wish there was no fight, just rational discussion lacking most of emotions. We humans have enough fighting these days already, in literal and every other sense.

This topic needs some quality discussion taking some time, so ideas permeate general public and folks have time to form an opinion. Not just be shock-pushed into an opinion by sudden dramatic arrival of something completely unknown and its sudden ubiquity, which can go very easily wrong even without nefarious parties taking advantage of situation, like they always do.

One wonders if some of the parties are using the sci-fi risk as a distraction from the more immediate one.

Personally, I think the creation of things like ChatGPT in a way poisons the well for future models to be created in the same way able to be up to date with future trends. I think within a decade much original, quality stuff will be more explicitly protected from free scraping and training. And at the same time, freely available stuff will become flooded with LLM-generated crap, so any "vaccuum up everything free" models of tomorrow will be trained more on today's model's output than on human output.

So whatever signifiers evolve to represent "this is real human-created stuff, not just AI crap" in conversations or art will be harder to adopt and copy without a lot of effort in your prompting and all.

Not just a distraction, but a great marketing tool. “ChatGPT is so intelligent that it poses an existential risk to humanity, try it for free today!”

Why not both? Seems to be a greatly profitable red herring...

Both camps have really weird blind spots that lead them to make rather counterproductive suggestions. One camp thinks about 'alignment' (which has serious risks if pursued naively). In the other camp it's the obsession with 'transparency' - which has its own risks.

Forget about Skynet. What happens when psycho militia takes the transparent training sets and transparent architecture and uses them to train its own AI for, say, creating some very innovative chemical weapons? Too many 'AI risk' things that the doomers think about 'work' almost as well if we merely have a non-AGI AI directed by known existing crazies. Maybe we should think about how much we really want to expose to the public.

Reply on news.ycombinator.com