Llama and ChatGPT Are Not Open-Source
spectrum.ieee.org
spectrum.ieee.org
https://en.wikipedia.org/wiki/Mark_Dingemanse
This co-author? Given the relevance of their academic background, I'd take their opinion on "the dangers of using LLMs released by the untrustworthy Meta" with a grain of salt.
If their academic background were relevant, they would have provided specific negative outcomes that may result from the usage of Llama 2, rather than a vague "the history of this company's choices does not inspire confidence".
What, of the many open source software releases by that company in the ML/AI field "does not inspire confidence"?
Didn't Facebook get it's start by Zucc populating it with his classmates without their knowledge or consent?
> Zuckerberg found himself brought before the Administrative Board for breaching security, violating copyrights and violating individuals’ privacy by using students’ online facebook photos without permission.
https://www.thecrimson.com/article/2004/6/10/mark-e-zuckerbe...
Title: Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators
Authors: Andreas Liesenfeld, Alianda Lopez, Mark Dingemanse
Abstract: Large language models that exhibit instruction-following behaviour represent one of the biggest recent upheavals in conversational interfaces, a trend in large part fuelled by the release of OpenAI's ChatGPT, a proprietary large language model for text generation fine-tuned through reinforcement learning from human feedback (LLM+RLHF). We review the risks of relying on proprietary software and survey the first crop of open-source projects of comparable architecture and functionality. The main contribution of this paper is to show that openness is differentiated, and to offer scientific documentation of degrees of openness in this fast-moving field. We evaluate projects in terms of openness of code, training data, model weights, RLHF data, licensing, scientific documentation, and access methods. We find that while there is a fast-growing list of projects billing themselves as 'open source', many inherit undocumented data of dubious legality, few share the all-important instruction-tuning (a key site where human annotation labour is involved), and careful scientific documentation is exceedingly rare. Degrees of openness are relevant to fairness and accountability at all points, from data collection and curation to model architecture, and from training and fine-tuning to release and deployment.
"For basic visitor statistics I use Matomo, an excellent open source alternative for Google Analytics. IP addresses are anonymized and data never leaves the server."
I don't want a sanitized LLM, and I don't have $60M lying around to train my own.
Copyrighted material, sexual content, political opinions, throw it all in and release it please!
Yes, reducing bias in the models is a noble goal, but introducing new bias and blindspots to do it is a no-no.
Maybe I just got added to a list somewhere for having this opinion.
It's also a necessary goal in order for these models to be more broadly adopted.
We've seen too many examples of bias in the training data set manifesting in ways that actively discriminate against people. Which is unethical and in many places illegal.
And having copyrighted material and sexual content in your model will simply open you up to lawsuits as is happening right now between authors and OpenAI. Not sure that is a position most startups want to be in.
As I mentioned, reducing bias is good so long as it doesn't introduce more bias elsewhere. The ultimate goal of course being a 100% bias free model.
Don't want your LLM to be used to create erotic fanfiction? Instead of gutting the LLM, just put a detector on the query and answers feeding into the LLM. Thankfully, with a non-neutered model you have access to a tool than can be used to perform such detection...
Which do broadly align with the values across most of the world.
But if you want to build something that has different values then go ahead.
But you can't expect companies to be complicit in doing something which is unethical or illegal.
Satire?
Using everything would be like taking a bunch of students final exams and then claiming the most common answers are the correct ones.
This isn't how expertise and accuracy works. Most things worth doing are not only genuinely hard and complicated but something that only a minority subset of accomplished people can do consistently well in.
Listening to everybody and incorporating their thoughts is only going to lead to wrong answers.
Being selective is the key here unless you genuinely want say, answers about to space to involve aliens and UFOs - because way more people believe in that then there are qualified PhD astrophysicists in the world.
Similarly, way more people believe in vaccine conspiracy theories then there are people with significant viral epidemiology backgrounds.
This pattern is true in every field.
As we have seen from history, there is not often an absolute truth to questions, only clusters of truths. We want our LLM to be able to perform reasoning, mathematics, and science, but expecting absolute truths in anything outside of those fields is a bit much. Wikipedia often takes a good approach here and strikes this balance well. You can represent the "crazies" and show their reasoning, and then draw attention to how it is commonly refuted. You don't get this if you just omit the "crazies" to begin with.
Edit: What I'm saying is that you can include/represent these adjacent clusters of answers instead of asserting one cluster to be true and allow the correct answer to be represented organically. And guess what? Some people will still disagree for whatever reason they have to disagree. We should be taking all this AI alignment money and putting it into education imo.
An expertise is needed from the reader when presented with false material. That's why so many people think parody news articles are real.
If you have ever gone through any comment thread on the internet about climate change you'll find a large volume of climate deniers citing God, greedy liberals, George Soros, whatever.
The people working with these fictions can always just add more fiction to fill in whatever hole. They aren't bound by reality or credulity.
Actually understanding global climate systems takes an incredible amount of study. You just can't blurt out a few paragraphs of words and get people to stop believing in Jewish weather machine conspiracies or whatever they're thinking.
We would have been done with that by now if this were the case.
Dislodging bullshit is hard. Humanity only started figuring out how a few hundred years ago and we're still fucking it up - such as in the replication crisis
If you're worried that there is more text on the internet saying that the Earth is flat than there is text saying that the Earth is round, then I understand where you are coming from, but then how would ANYONE come to believe that the Earth is round on their own? By learning how to reason.
Edit: My point is that if the LLM can't weed through bullshit properly then how can we expect it to get anything right at all? We can't just feed it a continuously updated list of facts that are true and facts that are false until the end of time.
Because it's not thinking, it's looking at probabilities. If you don't prune branches, you've created a disinformation radiator if that information is sufficiently populous within your corpus to be a pretty-good-bet.
The best analogy I've come up with is "it's like if you had an objective reality based on Wikipedia edit wars."
Some scams, like MLM, encourage massive volumes of this bullshit to be created and disseminated.
The democracy of information does not lead to an increased accuracy based on whatever information spreads the most. The internet is a multi-decade long global proof of this. There's a reason that classrooms have 1 teacher and say 24 students - only 4% of that room is to be trusted with the material.
To know if something is true, you need to eventually have some kind of external confirmation. Karl Popper wrote a lot about this. These chat systems can make predictions about the existing corpus but it can't like roll around town and do actual science to confirm things.
If all truth must be derived from self-contained systems than you have to make sure your self-contained system isn't bullshit.
A game for instance, can have its own rules and consistencies and things can be rationalized through the game. But how much that maps to a reality cannot be concluded strictly from inside the game.
For instance, if you want expertise in masonry and construction you need to make sure you aren't just feeding your AI Tetris and SimCity.
All that being said, who is to say that humans aren't just looking at probabilities when they think?
Humans will argue the right answer until their last days. It's frustrating how on-the-fence chatgpt can be. It's pretty interesting too, because in a professional environment one of the most important things you need to do is have an opinion and take a position otherwise you can't execute.
To English learners, that might be very difficult. I'm learning a new language now and I feel the pain. The textbook and instructor, in this case, is way more correct than the collective opinion of my fellow students.
The vast majority of things actually follow this pattern instead. Knowing what a programming function would output given a set of inputs isn't some gray area world of opinion.
When things are unknown, such as say Covid origin, again there's people way beyond my pay-grade doing investigative work here. Youtube comment threads and tiktok opinions are actually not as valuable as peer reviewed scholarship on the topic EVEN IF there's say, 10 youtube comments saying one thing and 5 papers in the Lancet saying another. The youtube comments don't become correct by volume here.
It's not some giant equal playing field.
As a linguistic descriptivist, hahahahahahahaha.
Linguistic descriptivism just means that, rather than hold up an ideal of a language and prescribe variants as wrong or right, linguists should simply describe the language based on its use. That doesn't mean that the language (or variants of it) has no grammatical rules. It just means that, rather than holding up some prestige variety as "the language," and unprestigious ones as "uneducated errors" that shouldn't be studied, that you study all of them and determine how they work and how they're developing.
No matter where you go, people speak languages with a limited set of phones, which are mapped onto morphemes, which combine by particular rules to form words, which themselves form larger groups, like phrases and sentences (often the line between these things isn't clear cut). But languages all have rules of their own.
To imply that descriptivism means languages have no rules would be like saying that physics has no laws, because a physicist makes empirical observations instead of just deciding whatever the laws of physics ought to be.
Even at the lowest level of phonemes to morphemes, accents introduce a huge amount of variety, and it doesn't really get more orderly as you increase the level of abstraction.
Or did I break some rules there?
We're not talking Oxford commas here. There are fundamental structures.
For instance, If I made that "are there fundamental structures" then it's a question.
Basic grammar is very real and not some subjective ephemeral thoughtstuff with wildly different opinions. It's so obvious that you don't even question it.
That's the way most things are - how to operate a faucet, close a window, put on a shirt, open a cabinet, use a fork... Most things in life aren't controversial.
Even the controversial ones like what's art and what's music, it's just a small periphery that's questioned. A symphony played by an orchestra is music, a painting in a museum is art - even these subjective categories substantially have extremely wide agreement.
True controversy is an outlier. (Ie, is 4'33" by John Cage music?)
You can claim whatever you want but with just things that's just exercising the freedom to be wrong.
I'm totally onboard with that.
At the same time, when talking about the physical world, most things are not like that. Humans for instance, get around by moving limbs called feet and not flapping feathered wings like a bird. This is the level of obvious I'm talking about.
The secondary point was that many things are this level of obvious and also specialized. For example: how does the animal cacomantis castaneiventris get around? Unless you have some kind of specialized knowledge you wouldn't know that's a type of bird.
The internet is awash in unspecialized people opining on specialized knowledge that is only controversial to the untrained eye in the same way that students don't get As on every exam they take. You need to actively work around people talking out of their bounds if you want your training data to be accurate
Could you expand on that a bit?
Why copyrighted material? Could we stop celebrating how tech is going to steal everyone's copyrighted works in a massive effort to replace the artists who made it? Why does everyone here hate artists so much? Do they not deserve any rights over their IP, eg, the right to say no when someone wants to make derivative works that replace them from it?
In all seriousness, they're afraid because they aren't very pleasant. AI will write things people will like and enjoy hearing. Preachy artists these days just make works they think people ought to want, and ought to like. Getting moralism out of art will be a great technological accomplishment.
More seriously: if you have this attitude for-reals and aren't Poe's Lawing me, perhaps you need a little more preaching aimed your way because your conception of art and the civic society is the closest thing I can think of to a humanities position being objectively wrong.
The ideological soma of agree-with-you AI being desirable is scary enough stuff. Removing communication from art being a "great technological accomplishment" is fucking dystopic.
While I love we can all immediately benefit from previous works, I feel for the countless artists and creators who's work has been integrated into models with zero compensation. There are already outstanding lawsuits seeking reparations, but this is new legal territory and there will be a lot to learn and decide in the future.
A society without copyright would be a much poorer one, and humans figured that out a long time ago.
That was before "Attention Is All You Need" and the subsequent work on LLMs that grew out of it. Things are different now.
Copyright may need to be updated to cater for the new world of AI but that doesn't mean as a concept it is evil.
https://www.theguardian.com/books/2023/jul/05/authors-file-a...
Whether demon or genie, any notion of putting this tech back in the bottle is a non-starter. As is turning it into a giant money grab for the copyright industry.
> Data being public is also notably not a license to use it for whatever purpose you want.
Under the Fair Use doctrine, it very well could be. It was when Google indexed every book they could buy, to the disdain of the Author's Guild and the titleholders they represented.
> Have some respect.
I will not respect an authority that forces me to rent digital content.
They are very obviously asking you to respect the individual artists.
LLM is a tool much like the internet is a tool. Yes, someone can use it to steal, but stealing is against the law.
Instead of encoding a criminal justice system into an LLM by omitting the possibility of stealing an artists work or omitting the knowledge of physics so someone can't learn how to build a bomb, we should instead just prosecute people for using it in that way intentionally.
How often do people get prosecuted for ripping off an artists style? The criminal justice system "hated artists" long before LLMs and it's not the responsibility of the tech. companies to rectify that in my opinion.
Even Open Source advocates don't really have the right to stop companies from using Open code. The license discourages it, but everyone from Tesla to Nintendo has been caught violating it's terms. Publishing stuff on the open web has always had consequences, unfortunately.
Ah yes, the famous massive tech corporation behind stable diffusion.
Key words being "from each other". AI only takes, it doesn't give anything back, it doesn't inspire. Their ultimate goal is to absorb the entire human history of art and then displace millions of people who received nothing from this transaction they were forced into, just so that some billionaire can afford another yacht.
I have no problems with AI models, but if you want to use art, writing, code, etc. for training, you should restrict your use to public domain works, ask for consent, or commission it. Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
Totally wrong. There is even a well known counter example: dream-like videos generated by ai wasn't something we had before. This statement tells me you never used it.
> just so that some billionaire can afford another yacht
You are heavily mistaken, that the scenario where ai learning isn't considered as fair use of material. In this case, only megacorps will be able to trains their own AI by spending billions in content.
> Obviously that would cost a lot of money so big tech companies are once again looking for a free ride.
You are constructing your own story while ignoring what is happening.
Adobe and Dall-e models were trained with datasets they mostly had rights over. OpenAI partnered with Shutterstock, and Adobe have their own photo stock. Big tech companies didn't had problem looking for image content. Stable Diffusion on the other hand is open source & open research, but didn't had any rights on most of the image they trained.
Copyright like many regulatory devices can be used for good, evil and everything in between.
Copyright has a fair-use exception for transformative works. It is difficult to look at the LLM's of the day and think "No, they have not taken the copywritten works and transformed them into something completely new."
There is no hate of artists. I don't know how you get from "It's not copyright violation" to "I hate artists". This is a matter of existing law. It is of course unsettled, the fair-use interpretation not yet tested in courts. But the same goes for affirming it's a violation of copyright.
These questions will have their day in court, and until then there is no need to make arguments in an inflammatory tone that that borders on personal attack. In the meantime, maybe engage in a conversation about how copyright law would need to be changed to account for this new technology, not condemn people who see a reasonable interpretation of existing law the differs from your own.
I think an argument could be made that it's fine for an LLM to learn from copyrighted works, but maybe it should have to go to the library to do it. Having your own human-accessible copy of those works (at home, or at Meta), doesn't sound as acceptable to me.
We phrase it like somehow the material is being copied into the LLM, but that’s not what it’s doing. It’s building a neural graph from the experience of consuming that content.
What would the world be like if humans couldn’t learn, train the weights of the interconnects of their neural tissue, from any material with a copyright?
At the very least I think LLMs trained on data that the trainer does not own or have rights to use in that manner should not be copyrightable.
My thinking “the enemy gate is down” when considering the tokens “Ender’s Game” is my recalling a learned association of those tokens to the given token string.
My knowing that doesn’t strip the copyright. My telling someone the meaning and context of the phrase generally doesn’t strip the copyright away from Orson Scott Card. I’m not reproducing his work but my knowledge of it. And it’s dependent on what I do with that knowledge and how if I’ve violated his copyright.
We are prosecuting the LLMs for possessing fragments of knowledge. And we’re assuming that the recall of some of those fragments means a copy of that work is in fact contained within the weights.
You blame "how tech is going to steal everyone copyrighted works", yet, they were already in the tech world: the internet.
It's generally consider ok to sell your fanart on comic festivals. But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?
A car and a pair of legs both move you from A point to B point. So why do laws treat automobiles and pedestrians so differently?
That's why I have put it in quotes.
> But do you think it's okay that Disney starts selling fanart of One Piece without the publisher's permission?
It's why it's called "fair use". Fanart isn't the only case of fair use. Parody for example allows it.
And AFAIK, no court yet have decided that AI "learning" is not fair use or what the limite are.
Why did you put one of those in quotes and not the other?
What other option would there be though? It seems like a binary field. You can either leave it wholly unaligned, or you can attempt to align the model, and you will introduce your own bias as a side effect.
In the future, and in fact in the present, there is already a "gray market" for unaligned models. There will surely be market for these and they will sell just any other item on this market.
“Bias” is not a unidimensional thing (or even one with a meaningful magnitude); you can’t reduce bias, you can make it more transparent (better documented) so that you can evaluate appropriateness for particular uses.
While I am not a big fan of the crew that pushes “alignment” as the main issue with regard to AI for other reasons, that’s a particularly good term for what is important – aligning the biases of the AI with the usage intent. “Reducing” bias makes sense only with regard to a specific usage intent.
Instead of all these complicated criteria, there are only 2 criteria that get to the heart of the question:
1. Can you run this on your own hardware
2. Can you use this directly or tweak this as a community to make porn without a bunch of barriers
This gets to the heart of the matter: Can you run the software without having to be dependent on the company, and can you modify the software to do something that the company may not want you to do with it.
If it is able to pass both tests, it is functionally "open source"
By that criteria, ChatGPT is definitely not open but Llama is closer
Is there another way to contribute?
I'm sure you could also offer a meta-comment on the process itself.
Right here the comment says they prefers Meta to keep the secret sauce as long as it allows they to (inderectly) access the copyrighted material etc. And the replies below are generally positive. So it's not a straw-man, at least in this thread, at all.
The entire subculture that this site takes its name from is hostile to IP.
I mean, except for the start-up founders that hang around here, but this site is mostly kept up as a way for them to recruit it seems like (and maybe a way to build good will).
Edit: To be clear, I'm not necessarily saying you're wrong, but you could probably pick a more sympathetic argument from a strategic perspective. What you've done is kinda the equivalent to going to the Vatican and arguing that something is a threat to idol worship. You might be right, but a good chunk of your audience wants that.
Open weights or something similar would be better though
https://en.wikipedia.org/wiki/Shannon%27s_source_coding_theo...
var a = b * 3 + 2;
You're telling me the * 3 + 2 part isn't code and only var a = b is code?
Also the article seems to be written by an LLM anywway, so it wouldn't matter in the end.