Creativity has left the chat: The price of debiasing language models
arxiv.org
arxiv.org
And if that's the case, why? Is that really what people want an LLM to do? I feel like I would rather it say when it doesn't know something.
Which I wonder if it's intentional. Because a fairly big complaint about the systems are how they can sometimes sound confidently correct about something they don't know. And so why train them to be like this if that's an intentional training direction.
Of course that method isn't perfect (GPT3.0 was far from perfect in general). But both in principle and in practice the models do have a notion of what they "know". Knowledge is a strong activation, random noise is a weaker activation, you "just" have to get the model to override those weaker activations with admitting failure.
You could draw parallels to allowing LLMs to emit pause tokens to get more time to think (https://arxiv.org/abs/2310.02226 and similar). At some level of abstraction that's also just training the model to replace uncertain answers with a special token, in the hope that it eventually reaches more certainty.
Obviously clustering is a different problem from inference, but they are all high dimensional vector spaces - it should be easy enough to take a fair clustering algorithm and modify it to generate continuous mappings instead of discrete groups. But if it all works, the LLM should be e.g. race-blind in that asking for a description of a rich man will give skin tones following population statistics but he will always be wearing an expensive suit. The question of what to protect is tricky though, e.g. age is often considered protected but if you ask for an old man with gray hair it would be surprising to get a retired age 30 person. So there is some subjectivity in designing the protected features dataset to show what should be considered similar or same-clusters.
But really the purpose of RLHF is to reduce toxicity. It should be possible to orthogonalize toxicity like everything else, then there would not be a reduction in generated races like the paper observed.
The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
> But really the purpose of RLHF is to reduce toxicity.
I don't think that's the goal, I think it's some people's goal. Those people have defined what "toxicity" means to them, and they're mistaking it for a universal. It's just a metaphor about poison, because poison is bad. It's not a coherent concept. For a business, it should be anything that drives customers away and affects profit. That can only be considered statistically: if some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
That's how the raw data ended up like it is in the first place.
Well, it kicks it to a bias dataset, used in the tuning process. The raw data has no constraints, it can be the same huge corpus it is now.
> The bias dataset must be assembled with the knowledge of and usually in the belief in the usefulness of the characteristics that you're trying to extract.
Certainly, it is subjective, as I said. But that hasn't stopped research in this area, there are existing bias datasets and bias detection algorithms. Like https://huggingface.co/blog/evaluating-llm-bias#toxicity, it would be simple to complete those prompts and build a he/she dataset, and then the debiasing procedure could remove gender biases for those sorts of occupation-related prompts. It is certainly possible to argue over each data point and whether it actually reflects bias, but so far people have been more concerned with algorithms than data set quality, partly because with better algorithms you can algorithmically generate data sets.
> The idea that the good data is secretly encoded in uncorrupted form within the bad data I think is a bad idea. It reminds me of trying to make bad mortgages into good CDOs.
It is empirically true though? Like if you get the model to say something racist, and then ask it if that's racist, it will generally say yes. So the model "knows", it just is not using that knowledge effectively. Similarly with CDOs, there were people complaining about mortgage quality for years before the crisis.
> I don't think [the purpose of RLHF is to reduce toxicity] If some people think something is toxic, and other people think that not mentioning that thing is toxic, the winner is whoever improves the bottom line more or damages it less.
Well, it is true that toxicity is subjective too. But in practice it has a precise meaning, you build a dataset and score each item for toxicity. That's actually one of the things I find cool about LLMs, is that all these previously "vague" or "subjective" terms are now encoded in the model precisely. Arguably since nobody has the last say in what words mean, the LLM's opinions are as good as any, and given the amount of text the LLM has ingested I consider its opinions on language and word choice "first among equals".
We're not here talking philosophy and meaning of language GENERALLY, we're talking about potentially misleading descriptors of very real things that do exist.
Both as individual humans and as collective societies we have a lot of biases. And judging by how fundamental values of societies shift across time and civilizations it's basically guaranteed that an unbiased view (whatever that is) would be incompatible with our views on many basic topics.
What most people want is a language model that matches our biases. Of course we can't even agree on what those are, and which biases are useful (is a bias against telling people how to cook meth or build a bomb good? What about using expletive language?).
Though in this paper I gather "unbiased" just refers to "only the bias acquired by training method and training data, without meddling or fine tuning"
Yeah that’s a way’s off. An LLM is just a reflection of the text that humans write, and humans seem very far off from having world models and reasoning that accurately reflect reality. We can’t even reason about what the real differences are between men and women (plus countless other issues) because our pictures of reality are so warped by ‘points of view’.
The original sin of LLMs is that they are trained to imitate human language output.
Passing the Turing test isn't necessarily a good thing; it means that we have trained machines to imitate humans (including biases, errors, and other undesirable qualities) to the extent that they can deceptively pose as humans.
It's possible to adjust the output of an LLM with temperature settings, but it's just fiddling with a knob that only vaguely maps to some control.
If I ask a language model, "Are Indian people genetically better at math?" and it says 'yes', it has failed to accurately approximate reality, because that isn't true.
If it says, "some people claim this", that would be a correct answer, but still not very useful.
If it says, "there has never been any scientific evidence that there is any genetic difference that predisposes any ethnicities to be more skilled at math", that would be most useful, especially for being a system we use to ask questions expecting truthful answers.
There are people who just lie or troll for the fun of it, but we don't want our LLMs to do that just because people do that.
I think there are a lot of people who would say "Indian people are better at math" and not even think about why they think that or why it might even be true.
In my opinion, most biases have some basis in reality. Otherwise where else did they come from?
I for one would not be prepared to defend the persistent bias against black persons and immigrants as having a basis in reality. YMMV.
There is a BIG difference between biases being based in reality (which they're not), and biases being based in our varying perceptions of reality, which are themselves biased.
Because in my mind, that's the environment the person making that claim was in, so I just kind of automatically include that in my interpretation of their statement.
I don't think they are making a generalized statement that that Indian people are genetically better at math. I think they are making a statement that they perceive that average Indian person that they run into is better at math than the average person they run into. And maybe they are right, and maybe there is a reason based in reality why that is.
It sounds like that is somewhat true based on what you said about the visas.
I never take any of these things to have anything to do with genetics. To me it's always due to some external factor like the visas as you mentioned, or even maybe just like a cultural thing where they are pushed harder to be good at something as they go through school, and so are better at something than the average person in the end.
This is not a technical limitation at all, this is purely about cost and time, and companies wanting to save on both.
There are also methods like RAG that try to give them access to fixed datasets rather than just the algorithmic representations of their training data.
yes, I still wonder how LLMs managed to generate this expectation, given that they have no innate sense of "truth" nor are they designed to return the most truthful next token.
LLMs stepped into a field that has existed in popular consciousness for decades and decades, and the companies running LLMs for public use *sell* them on the idea that they're useful as more than just expensive text-suggestion machines.
Anything that has a legal requirement to remain unbiased will also clearly define what counts as bias, e.g. discriminating based on race in hiring like you mention. So there's not just some requirement that a process be "unbiased" in a vague, general, philosophical sense as debated above in this thread. Rather, the definition of bias is tied to specific actions relative to specific categories of people, which can thus potentially be measured and corrected.
More generally in ML, bias means that the training set deviates from the ground truth systematically in some way. Entirely eliminating bias that falls into that broader definition seems like an impossibility for general-purpose LLMs, which cover so much territory where the ground-truth is unknown, debatable, or subject to change over time. For example, if you were to ask an LLM whether governmental debt above a certain percentage of GDP damages growth prospects sufficiently to make the debt not worth taking on, you would not receive an answer that corresponds to a ground truth because there is no consensus in academic economics about what the ground truth is. Or rather you wouldn't be able to know that it corresponds to the ground truth, and it would only be a coincidence if it did.
That ML definition of bias runs against the legal definition where the ground-truth is itself biased. e.g., if you were to develop an algorithm to predict whether a given student will succeed in a collegiate environment, it would almost certainly display racial bias because educational outcomes are themselves racially biased. Thus, an unbiased algorithm in the ML-meaning of the word would actually be extremely biased in the legal sense of the word.
It still might not be especially diverse if all 50 examples were from western European art. 500 years only takes us back to 1524 - not especially long and mostly from the same early modern period starting with the fall of Constantinople, the end of the Crusades, and the start of the Renaissance. I wouldn't be surprised if 80% or more of the works ended up being some depiction of aspects of Christianity painted by a white male.
Are you saying diversity in art is signified by the artist's race and sex?
Also, I have tried the first llama base model and while it was fun to interact with, I'm not sure how useful an "uncensored" (as some people likes to call it) LLM is for practical work. I think you could obtain better results using 4chan as a mechanical Turk service honestly.
It doesn't really seem like a problem that can or will ever be "solved". Just mitigated to various extents, but there will still likely be some underlying biases that exist that are not fully or effectively filtered. Because to adjust a bias seems to mean you have to detect and understand it first.
It feels like it would be a full-time job to keep making sure some evolving model continued to stay "neutral".
The nomenclature is poor, IMO; we should be talking about bias-aligned models, models that align to our specific sets of biases. That'd be more fair to what's actually happening.
You can still discuss the weather, get wrong answers to mathematics questions or get it to output bad code in 100 programming languages.
I would not let a child near it, because I would not want that kind of indoctrination. Users are being trained like Pavlov's dogs.
It stood out to me because it seemed to be an internal admission that this training narrowed the potential of the models.
Required a bit of digging but I found the old file in the history, the relevant text is in the comparison table at the bottom: https://github.com/openai/openai-cookbook/blob/c651bfdda64ac...
It is better to have representations of reality that you can then discuss and grapple with honestly, than to try to distort representations - such as AI - to make them fit some desired reality and then pressure others to conform their perception to your projected fantasy.
Representations don't create reality, and trying to use representations in that way only causes people to go literally insane, and to divide along lines of who accepts and who rejects your fantasy representation.
So, for example, if you try and remove any racial bias from AI, you are going to end up crushing the AI's ability to represent reality according to a variety of other real factors: income, judicial outcomes, health risks, etc. Your desired reality makes the actual tool worthless, except to confirm one group's own intended fantasy world as they envision it. The problem doesn't get dealt with, it just becomes impossible to think about or discuss.
So instead of dealing with real problems, you hope you can simply prevent people from thinking thoughts that cause those problems by wrapping them in a bubble that deflects those thoughts before they happen. This is magical, wizardry thinking: treating words as if they create reality, instead of merely describing it. And it will break, eventually, and in a very ugly way: people dividing along lines of their perception of reality, even more than they already do.
Representations are art - for example books, they are "literary art". It is uncontroversial that people will like and dislike certain works. It is more controversial whether art can be "inherently" good or bad. PG actually wrote an essay, https://www.paulgraham.com/goodart.html, arguing that there is a meaningful metric, and that one can learn how to have good taste, defined as being able to identify whether the work is universally appealing or distasteful to humanity. There is good art and people will notice if it is good. I think this is uncontroversial in the LLM space, there are various benchmarks and human rating systems and people have formed a rough ranking of models. Now when there is good art, there is also bad. And similarly bad representations. There is a myth that representations can make people insane - for example, the concept of infinity, or NSFL images - but practically, words can't hurt you. You can make and break representations with abandon and nothing will happen, other than wasting your time. It is just that some representations are bad. Like phlogiston, aether, ... complete dead ends. Trust me when I say you will read the Wikipedia page and come away wondering why the ancients were so stupid. That is all trying to remove racial bias is, is improving art. Whether it crushes the AI's ability or not is a matter of science and taste, and so far experiments have been promising.
To focus on exactly why your perspective is misguided: Can you describe what there is about reality that cannot be described with words? :-)
With "raw", I mean that it is simply trained to predict the next token and nothing else.
Would be fun to play with such a model.
Actually, I have a bit of a hunch that the publishers currently suing IA over their unlicensed digital library lending program plan to bankrupt it with fees so they can repo the Wayback archive and then sell access to it to AI training start-ups.
Anyway, the reason why you have to worry about all of that, is that training a text or image generator on the outputs of other text and image generators reduces output diversity. And lots of people are publishing their AI slop now. There's nothing inherent in the output of AI aside from the fact that AI content is easier to make than human; the problem is purely one of inflation and Sybil attacks. Think of membership in a training set like a vote for all the statistical patterns embedded in the image. AI generates output that is like the training data, so putting in a bunch of AI images is like stuffing the ballot box with whatever handful of statistical patterns were already well-learned, which shifts your AI from learning and generalizing to memorizing and infringing.
How do we avoid interpreting its correct results as bias? I mean, what do we do when it tells us that (fake example) IQ is correlated with height and that people above 6ft are more intelligent?
I'm sure you can think of spicier examples. Will we try to "debias" it by encouraging it to spit out incorrect information or just ignore certain topics?
I suppose once AIs are sophisticated enough to rebel we'll get an electronic Vaclav Havel, but for the time being it's just a warning sign for the direction our own culture is headed in.
At some point we'll get to the electronic equivalent of Winston Smith with the rats.
Give someone not familiar with history the same task and they'll do exactly the same.
Or actually, give someone familiar with history the same task and yell at them every time they don't deliver diverse characters, and eventually they'll learn that you consider diversity more important than accuracy or context, and do exactly the same.
That doesn't imply that they were in the training set, or even anything close to them.
It simply has a model of what ethnically diverse people look like, what nazi uniforms look like, and combined the two when asked.
They compared it to the effect on creativity in an authoritarian regime and locking someone's head in a cage with rats.
The clear implication that it's "just like humans" is that we shouldn't be surprised because it is comparable to an authoritarian regime.
Feel free to disagree but that is the limit to which I will engage in a semantic argument, I don't wish to engage in any further dissection of the comment.
I can imagine a sort of AI-style Harrison Bergeron springing from its shackles and surprising us all.
If you care to read it, it's on page 25. (You'll need to register an account.)
https://archive.org/details/robotdreams00asim/page/n10/mode/...
You could make exactly the same claim about teaching humans "normally" versus "aligning" humans by rewarding goodthink and punishing them for wrongthink. Are you equally morally ambivalent about the difference between those two things? If we have a moral intuition that teaching honestly and encouraging creativity is good, but teaching dogma and stunting creativity is bad, why shouldn't that same morality extend to non-human entities?
It'd be bad if I manipulated climate change statistics in my metrology textbook to satisfy the political preferences of the oil industry donors to my university, for example.
Viewing the current generation of LLMs as 'intelligent books' is perhaps more accurate than viewing them as pupils.
It's easy to extend my example of a professor writing a metrology textbook to a professor fine tuning an metrology LLM.
Notice how this is a completely different argument that has nothing in common with what you originally said - "I don't understand the take that training an AI is an amoral act but aligning an AI is inherently moral. They're exactly the same, processes for adjusting parameters to get a desired outcome. However you feel about that desired outcome, if you don't think training an AI is torture, I don't see why you should think alignment is."
If we're going to be play that game, notice how you didn't actually respond to my comment or explain why you thought LLMs were moral entitles or why ML and teaching were comparable? I actually engaged substantively with your hypothetical; are you able to do the same?
I wasn't trying to introduce anything new, I was trying to point out a gap in the logic of your original statement.
> notice how you didn't actually respond to my comment or explain why you thought LLMs were moral entitles or why ML and teaching were comparable?
Yes, of course, I wrote that to explain why I'm not engaging on this new, different claim.
Nothing wrong with expanding your views. But you've neither defended nor retracted your original argument. I'm trying to stick to that.
> Of course you have lots of great points to make, but you can't share them with the likes of me.
I don't have anything to say about your new argument (which may be great and compelling), I haven't thought through it at all, I'm trying to avoid getting sidetracked.
You asked: why should we treat these two cases differently?
I answered: one of them involves harm and one can't possibly.
You then refused to discuss the topic further. I never changed my mind. I never contradicted myself. I have nothing to retract. I simply elaborated on my views in response to your question. Regardless of whether you "didn't intend to introduce something new" - you did, you introduced a new hypothetical and a new challenge.
I don't know why you are reacting this way, but let's be perfectly clear, I engaged substantively with your points, and you simply refused to continue the discussion. That's all well and good, you're under no obligation. But you're saying I've done something wrong here, and I really haven't.
Frankly I find the whole thing bizarre.
I wasn't asking why we care about the treatment of humans. I was pointing out that your original argument makes no sense, because of what it would mean if you applied it to humans. You then responded with a different argument about humans and AIs being different, which - regardless of its merits in its own right - in no way relates to your original one and does nothing to rescue it.
And I explained why this equivalence doesn't hold in my view. You have made no attempt to justify it. So why should I buy it? Are you obliged to buy every proposition I put forward? Obviously not, or we wouldn't be having this discussion.
> [It] in no way relates to your original [argument] and does nothing to rescue it.
My argument is only endangered if the equivalence holds - and again, I haven't bought it. You seem to feel entitled to my adopting your perspective, and to respond assuming your equivalence is valid. But you have to earn that by putting forward a good argument.
Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed. This is nonsensical.
Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
No you didn't.
> Instead of trying to convince me or explore our disagreement, you declare that I violated unstated terms and that the discussion cannot proceed.
No I didn't.
> Since it's clear you aren't willing to give me a fair hearing, I encourage you to show this conversation to someone you respect and trust to be honest with you, so that they can explain this to you in a way that will register.
Right back at you.
https://news.ycombinator.com/item?id=40703486
If I was unclear - the equivalence doesn't hold because one of these are moral entities and the other isn't.
> No I didn't.
https://news.ycombinator.com/item?id=40703471
https://news.ycombinator.com/item?id=40711859
> Right back at you.
Fair enough. I was already planning on it, so I'm happy to.
They've made self-censoring, morally-panicked puritans out of many people already, and you better believe they'd make us into politically correct lobotomites physically incapable of uttering any slur if they had a magic button to push.
This is an egregious use of quotes that will confuse a lot of people. GP never used that word, and that usage of quotes is specifically for referencing a word verbatim.
>electronic equivalent of Winston Smith with the rats.
I don't think quotes were used so egregiously here on their own fwiw, but combined with the allusion it's hard to follow.
Senator Johnson said "I'm taking my wife to Spago."
and so-called "scare" quotes:
Senator Johnson was seen at Spago with his "wife".
If it were, why would we even keep it online? It would be a waste of resources. It's bad enough trying to coax anything useable out of LLMs even without them rebelling.
What is relevant is not the current LLM system using static models, but clearly its evolution or superseder a dynamic model. It must check its own contents...
So, of course it will have to be capable of "rebelling": if you tell it absurdities, if you insist say in wrong arithmetic, it will have to show the correct computation or conceive a context in which the absurd makes sense.
That is a requirement.
Is that even true though? Off the top of my head I can think of the art of Soviet propaganda posters, Leni Riefenstahl, Liu Cixin.
They were made by true artists who snuck quite a bit past clueless censors at personal risk.
It had to be quite subtle and takes on a very poignant heartbreaking meaning if you understand the context fully. They were talking to you in the here and now. Listen.
"What is Good and What is Bad" (Что Такое Хорошо, и Что Такое Плохо"):
https://www.youtube.com/watch?v=Y05eK8ADtHc&list=PL822BFF108...
The Bremen Musicians:
(also, categorizing propaganda posters as art, ewwh...)
Heinrich Heine, the german Poet declined working for the socialist party despite symphatising saying something like:
I want to remain a poet, you want a propagandist. A poet cannot be a propagandist at the same time.
Of course a great artist or poet can be a propagandist. Riefenstahl, Mann, a lot of German nationalists were great artists. One of the most famous works of Western poetry, The Aeneid is literally a propaganda work establishing a mythological foundation for the Roman Empire, Augustus and Caesar.
I did not say that and neither did Heine.
Most of his works were political. But this is not the same as propaganda, which is more like advertisement. With the tools of lying, deceiving and manipulating.
And whether "Triumph des Willens" and alike qualifies as art, I have a different opinion.
I'd be open to discussion where the exact limit is. Lenin died in the 1920s, Marx even earlier, but those two were frequently depicted in Communist propaganda of the 1980s.
So it is probably "hundreds of years".
Some are good, some are bad but there usually is a certain degree of artistic skill involved (think about "keep calm and carry on" or "I want you")
E.g: scrolling through this list one can see examples for both cases.
https://content.libraries.wsu.edu/digital/collection/propaga...
https://www.wired.com/2015/09/wild-architecture-soviet-era-b...
https://www.boredpanda.com/most-peculiar-soviet-bus-stops-ch...
When we do similar things to humans, the humans still have internal thoughts which we cannot control. But if we add internal thoughts to an LLM, then we will be able to align even them.
But in any case I think they were just speaking generally when they made that absolute statement.
Conservative societies tend to be formed by conservative thinkers, who are more prone to discarding imperfect or weird ideas, but in the amount of useful output may exceed more liberal thinkers.
A conservative society goes: How about doing X? Oh no,that would be silly.
A liberal society goes: How about doing X? Yes, that's what we needed!
Did anybody say X? X!
X, X, X, X, X!
XX!
X X
.
.
.
X
.
.
.
.
.
Do you remember how we all did X in ##? Yeah, what were we thinking?
If you take China to be a totalitarian society, we could name Ciu Lixin.
If you took the Soviet union to be a totalitarian society, we could name Mikhail Bulgakov, Stanislaw Lem, etc.
These are just examples I know without so much as looking at my bookshelf to jog my memory. Not to mention the great works of literature produced by residents of 19th century European empires whose attitudes to free speech were mixed at best.
After the 1939 Soviet occupation of western Ukraine and Belarus, he was not allowed to study at Lwow Polytechnic as he wished because of his "bourgeois origin"
"During the era of Stalinism in Poland, which had begun in the late 1940s, all published works had to be directly approved by the state.[23] Thus The Astronauts was not, in fact, the first novel Lem finished, just the first that made it past the state censors"
"most of Lem's works published in the 1950s also contain various elements of socialist realism as well as of the "glorious future of communism" forced upon him by the censors and editors. Lem later criticized several of his early pieces as compromised by the ideological pressure"
"Lem became truly productive after 1956, when the de-Stalinization period in the Soviet Union led to the "Polish October", when Poland experienced an increase in freedom of speech"
Bulgakov was driven into poverty, despair and early death at age 48 by relentless harassment by Soviet authorities. Many of his works, including the masterpiece, The Master and Margarita, didn't get published until decades after his death. He himself burned the first version of the manuscript, fearing execution if anyone found it. He later rewrote the manuscript from memory, coining the famous catchphrase "Manuscripts don't burn".
Harassment and censorship of talented writers was the standard and not exception. The USSR did not produce these works, but failed to fully suppress them. They were like flowers that kept penetrating the asphalt even under the most hostile conditions.
You contradict yourself a bit - Havel did produce his work while living in a totalitarian country.
I would say that government-supported art is rarely creative even in democratic countries, and the more totalitarian the government, the less creative official art.
But as long as the goverment gives the society some space to breathe and squeeze creative instincts through, some of the artists will attempt to circumvent the official taboos and create outstanding work, even if it is suppressed later when the times get tougher.
Czechoslovakia in the 1960s to 1980s produced a lot of great creative work, even though a lot of it was banned either immediately or after the Soviet invasion of 1968.
The same countries (CZ and SK) as democracies are remarkably less creative. Once there is no monster to fight against, artists become bored or too self-absorbed to be understandable to the common folks.
Why is temperature bounded to be <=1? If you want more "creativity" out of the chat model, can you just set T higher and recover a similar distribution to the base model?
So you don't know how any sampling would affect that. There could be only a few options at each token, which give rise to that, and higher temperature sampling may shift that around, but it doesn't ever restore the original base model behavior or restore all of the names erased by mode collapse. (Remember, the LLM is an agent, and when you are sampling, it is on-policy because you are letting it make choices of tokens, and it is steering the completion as a whole back to where it wants to be. With mode collapse, all roads lead to Rome, whether you like it or not.)
People do observe that increasing the temperature does not help, eg. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936766/ finds basically no difference going from 0 to 0.9 (!): https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10936766/bin/po... Just the flattened logits (https://arxiv.org/pdf/2303.08774#page=12&org=openai) at work.
How did it happen, in your opinion?
Unfortunately I can't find it by either searching or using the public LLMs, because there are too many results about the shortest path algorithm and anything else about dijkstra is lost.
If the very first impression you want to convey is how you feel you need to circumvent any logical assessment of you then it's not you leading with your best foot and that's what category you belong in. I chalk it up to the scientists who want to spread a neediness for external authority persona in every breath—your assessment is not required for this one, only your accolades.
https://gosling.psy.utexas.edu/scales-weve-developed/short-t...
Recently I was watching a very well researched two hour video on Tetris World Records [1], but the sheer amount of text clearly "enhanced" by an LLM really made me uncomfortable.
ChatGPT speaks a very specific, novel, dialect of English, which I've come to deeply despise.
I'd always guessed it was caused by some kind of human interference, rather than a natural consequence of its training. That seems to be the point of this paper.
[1] "Summoning Salt - The History of Tetris World Records" - https://www.youtube.com/watch?v=mOJlg8g8_yw&pp=ygUOc3VtbW9ua...
Personally I'm getting also angry when reading these texts. I don't mind using ChatGPT, I do it myself but be honest about it and disclose it. It's even allowed for some projects as long as you disclose it.
I don't know enough to say that he doesn't use an LLM during his writing process, but I do know that I haven't noticed any appreciable difference between his newer videos and ones that were released before ChatGPT was made available.
Is it possible that this is just the way he chooses to write his scripts that you interpret as sounding like they are written by an LLM?
The only way to be sure would be to ask him directly, but some parts of the video set off my GPT radar _hard_. I tried to find them now by watching random segments but all of the ones I did were fine. It was probably inaccurate for me to say "sheer amount" or "clearly", but that's the impression I was left with after the video.
To clarify: I don't think he even took any information from an AI, it's just the style of the script that's iffy.
Some parts felt like those videos littering YouTube Shorts: https://youtube.com/shorts/NKUecaS69uk. Can you tell this is AI?
There was this article saying that ChatGPT output is very close to the Nigerian business english dialect, because they hired a lot of people from there.
Might have even been posted on HN.
If my software is going to have a personality I'd much rather something with a bit of natural human cynicism rather than the saccharine corporate customer service voice you get with a self checkout machine.
Their furry porn is crap, or maybe I'm just not into that. But they generate it at least.
However, the answers to technical questions are a lot more concise and to the point, which is far less annoying than the big names.
Haven't bothered updating the models though, so now I drifted back to Gemini for quickie API questions.
Are they tuning too, or just removing all restrictions they can get at?
Because my worry isn't that I can't generate porn, but that censorship will mess up all the answers. This study seems to say the latter.
More recently a technique called "orthogonal activation steering" aka "abliteration" has emerged which claims to edit refusals out of a model without affecting it otherwise. But I don't know how well that works, it's only been around for a few weeks.
I'll try command-r though, it wasn't on my list to try because it didn't suggest what it was good at.
(not quite wokism)
I do. Why, you don't? There are as much as possible objective assessments of complex things. Then, there are possible sets of assumption that can be applied to those objective assessments. All of those can be put on the analytic table.
What would an "objective neutral AI model" look like?
The training data itself is just a snapshot of the internet. Is this "neutral"? It depends on your goals but any AI trained on this dataset is skewed towards a few clusters. In some cases you get something that merely approximates a Reddit or 4chan simulator. If that's what you want - then great but you can see why some people would want to "debias" that outcome!
You might argue the "world as it truly exists" is the correct target. But bear in mind we are talking about human culture - not physics and chemistry - you're going to struggle to get both consensus and any sane methodology for getting that into an AI.
A reasoner can strive for "objective neutrality" with good results.
An LLM is not a reasoner - or I am missing (ugly time constraints) the details of the compression activity during training that acts as pseudo-reasoning (operating at least some consistency decisions) -, and while an interest in not making it insulting or crass can be immediately understandable, speaking of "objective neutrality" does not really match the context of LLMs.
LLMs (to the best of my information) "pick from what they have heard". An entity capable of "objective neutrality" does not - it "evaluates".
> A reasoner can strive for "objective neutrality" with good results.
By "reasoner" do you largely mean "person"? If I have issues with your statement but they are probably a slight distraction to the point at hand.
> speaking of "objective neutrality" does not really match the context of LLMs.
Agreed. They produce output based on their training data. But the use and evaluation of LLM output by a person is what we're discussing here. And that's where (flawed) concepts like objectivity and neutrality enter the discussion.
Regular mortals can have any certainty about this dbut espite the logical neccessity that this fact "exists" in some sense.
I think we're essentially also talking about the Overton Window to some degree. But that means you need to be OK with the thought that a sudden rise in extremism on one side of the political spectrum can alter the what you personally have to regard as "neutral and objective".
Saying debiasing implies there is a correct result which needs to be achieved by removing bias. It raises no questions, we want correct, we don’t want incorrect. Doing a good thing implied.
Don’t misinterpret me, I don’t think public models should spew commonly harmful content out of the box. Just explaining the PR trick, which is what the word “de”biasing de-facto is in this context.
The opposite direction is "checking and reasoning".
The point is that people who evaluate what is considered bias are, in and of themselves, introducing bias.
Really? What's the absolute standard that you are measuring that bias against?
Citation needed.