Is a truly open source 2 trillion token model even possible?
Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses.
Even if Meta released this under Apache 2.0, there's the sticky question of the training data licenses.
You need to pay Disney royalties out of every paycheck because you watched Toy Story and Star Wars. These movies updated your own neural weights.
If the AI has seen Star Wars and that's enough to find it liable, then you should be too.
If the AI has seen Star Wars to understand science fiction and modern culture, then it's no different from us or any other artist.
YouTube zaps videos that contain too much copyrighted stuff for this very reason.
It's an exact moral equivalent to learning. I wouldn't expect PyTorch to be running the citric acid cycle under the hood, and that shouldn't matter.
> Copilot reproducing `Q_rsqrt` verbatim, including the magic numbers, formatting and comments.
Humans can recite lots of things verbatim. Song lyrics, Shakespeare, digits of pi, Bible verses, you name it. We're so early, and the models are only one little engineering trick away from not doing this unless requested. We'll probably be able to tackle this at the level of the weights themselves.
> copyright law, which is designed to protect the human creative process only.
This is the point. Does AI contribute positively to the economy and the human condition? Does it outperform the old status quo? We're going to find out, and I'd expect for the law to fit to the outcome it views as optimal.
I'm betting that the law will view this similar to industrialization. That more people will be enabled through AI than inconvenienced, that more works will be created, and that more money will move and flow than before.