Geoffrey Hinton: Open sourcing AI models akin to open sourcing nuclear weapons

So does he think large, private, for-profit companies should be in charge of nuclear weapons as well?

It’s silly, but actually not an terrible analogy because nuclear weapons aren’t really that complicated as a technology — there’s even somewhat famous caselaw established when a magazine attempted to publish the basics. [1] While it’s not terribly technologically complicated, it is very complex—to the point that it’s only feasible for nation-state level actors. It requires huge investments over lengthy time to develop nuclear weapons…kinda like AI models. (The analogy breaks down at this point because there’s a huge difference in scale—AI costs millions; nuclear weapons cost billions.)

[1] https://progressive.org/magazine/november-1979-issue/

I'm pretty sure nuclear bombs are open source. https://en.m.wikipedia.org/wiki/John_Aristotle_Phillips

This people vastly and ridiculously over-estimate the capabilities of LLM. LLMs capabilities are deceptively impressive, and while very useful tools, probably they are the peak of their potential and unless augmented by something else, will plateau in the not so far future.

Besides that, it is not like building LLMs is a new manhattan project which a small cadre of very few selected scientists able to do it. If anything, the biggest constraint for any one building a state of art LLM is access to massive quantitities of hardware and training material.

The level of hype about LLMs is becoming frankly ridiculous.

Self-serving, manipulative, false, and offensive.

I have to say it.

"You wouldn't download a nuke..."

his intentions are good his reasons are sound but ultimately he's probably helping the wrong people

I mean, by the same token:

> In 2017, he co-founded and became the chief scientific advisor of the Vector Institute in Toronto

> The Vector Institute is a private, non-profit artificial intelligence research institute in Toronto focusing primarily on machine learning and deep learning research.

If a random Canadian non-profit was developing nuclear weapons, I think, er, something would probably have to be done about that, too.

ehhhhh....fuck it...let's see what happens...