Hacker News Reader

Comment by pksebben | Hacker News Reader

If we really want to curb the potential for harm from these models, open them up. Share the checkpoints and the code to train them. Democratize the development of new systems.

The infosec world figured this out ages ago. Yes, it's possible that some bad actors will gain access to capabilities that they didn't have before - but the strength of a billion eyes is huge, and can rapidly accelerate solutions to deficits.

What we need is a massive, open, community curated dataset that continually evolves. Craft regulations that enforce data availability and free access. Without such mechanisms, AI systems all have a SPOF that's just waiting to bite the collective us in the ass.

View on HN

We should also make all nuclear weapons blueprints open source on github- We'll be much safer if everyone can take a look at them!

the analogy is more like nuclear physics should be opened up, which it has. (note: I'm not arguing for or against open models, but AI is not analogous to nuclear weapons. AI would be more analogous to nuclear physics, and skynet to a nuclear bomb)

and with many eyes we'll be able find any bugs! :D

I can see a kickstarter in the near future.

micro drone swarms will eventually be much more effective than a nuclear weapon, less collateral damage and no radioactivity afterwards

Getting shot by a micro drone sounds painful- I'm at least hoping the the AI Yann and Andrew help create will be smarter than that, and will have found a less painful way to kill us

It will be more like a pin prick or a grenade. The tech is basically here already and we don't need AGI to realize the potential. Just look at what they are doing with drones in Ukraine to see where things are headed in warfare. Now pack them into a cluster bomb like delivery system and tell them to kill anything that looks like a human.

The information required to build one is already available - albiet not a highly optimized one. How to refine uranium, the design, etc. You can even buy uranium on amazon legally.

The main barrier to entry for them is the massive amount of refinement required to make any useful amount of the required materials, like multiple warehouses size operation.

---

The big difference here is stable diffusion has shown you can train a highly capable model under 1 million dollars. LLaMa is making some interesting advances as well (finetuning a LOrA takes ~3 hours on a 4090). Training/finetuning/inference capable AI is quickly becoming more feasible (albiet trailing the commercial stuff) for the average consumer hardware.

It isn't the potential harm, it is the real harm to people all over the world as their jobs are displaced at rates that are too high for the economy to refactor itself.

Can growth be too fast?

This is kind of my point - there isn't any slowing or stopping progress on this. We have effectively two paths

- development behind walled gardens and closed doors, which will accelerate the wealth gap to war- and poverty- inducing levels

- access to this powerful game changer for the people who would otherwise be losing their purpose in the social structure. There's no reason your job has to evaporate as an artist, or programmer, or anything else if you can just keep doing the same thing empowered by a cheap, accessible force multiplier like AI.

It really comes down to whether or not the closed models are _enough better_ that they can start charging exuberant rents to people so they can effectively continue contributing to society.

I am talking about growth, not who controls it. Yours is an extremely important distinction, but not entirely related my point of too much change to quickly. Even with "just" ChatGPT 3.5 performance, we are upending everything in months.

The growth is here. Whether the models become openly accessible is immaterial to that fact - closing them up and burying them in regulation won't do anything to slow the progress down; it will just slow progress down for the likes of you and I.

The paths I see, in broad strokes:

- models become open and easy to hack on. We all lose our jobs. But we can now find new jobs using these models to do similar work.

- models stay closed and only get developed by BigCo. We all lose our jobs. Remediation for this development may or may not happen and who knows what it will look like.

The core difference I think you and I are dancing around here is a belief in our collective ability to hit the brakes on progress. I understand the desire to do so, but you won't stop a river by putting a rock in the middle and I think this is a situation with that level of momentum.

I seriously hope we don't lean heavily on knowledge from the infosec industry to try to secure potential AGIs which seem completely different from anything the infosec industry has any experience handling.

When a dangerous exploit is discovered, best practice is to give good actors playing defense a heads up and time to prepare before publicly revealing the exploit, and preferably only revealing the exploit once critical systems have been hardened against it.

Cheap general AI is a potential exploit of everything everywhere all the time. No one is prepared for it.

A billion eyes are powerful. Giving every actor an eye-making machine is uncharted territory. And opening it up is a one-way gate - you can always delay opening it up another day, but you can't un-open it once you've let it out. So you should be very, very certain you're prepared to commit yourself, everyone you know, and the entire human race to living in that world before you make it so.

who is “we”? it doesn’t work like that

Reply on news.ycombinator.com