The problem is chatgpt always answers. It's never, i don't know. So when you ask for five examples, or libraries it doesn't say I don't know. It will just generate the most likely ones. Which is a big issue in this case, but useful in most.
It in fact never knows. It is always guessing and BSing. It is also very good at it, better than any human, so the BS is quite frequently correct. But it doesn't know.
Which of course always prompts a lot of psuedo-intellectual chin stroking about "well, what is knowing anyhow?" I don't have to answer that question to point out that what we want and what GPT provides aren't the same thing, nor is what GPT provides and what we think we're getting. That is sufficient for there to be a problem.
I believe AIs will have a satisfactory concept of "knowing" someday. Some may even exist today. But such AIs will have to incorporate language models as a part of the system, not have language models be the system. Language models can't do the thing we really want, nor the thing we think we're getting.
For kids story writing I've been getting better results with 3.5 at times.
Where 4 is way better af coding.
I have a small library of children's books and we've read them all several times, the good ones many times.
That said, I wouldn't personally turn to these language models. From what I've seen they tend to generate rather bland and boring stories. I would rather make up my own or reread "Kackel i grönsakslandet" for the hundredth time.
The fact that RLHF broke the calibration comes from the GPT-4 paper, possibly the only interesting technical detail that they include.
In the UK, there is a competition called the "junior maths challenge", or something, which is a multiple choice quiz where correct answers are +1 and incorrect answers are -6 (so guessing has negative EV). I think we need a similar scoring system here.
I'm 100% positive that if the guard rails that OpenAI put on were taken off of ChatGPT it, for instance, would not be a big fan of jews given the width and breadth of antisemitism online (and certainly in its training set).
clouds are white
crows are black
swans are white
Of course LLMs have a much larger and more complex context than the single token in my example. But if the training data contains many news stories about professors being accused of sexual misconduct (which is newsworthy), and few news stories about professors behaving with propriety (which is not), then when querying the model for a story about a professor then it is likely to reproduce the statistical properties of its training data.
It is a writer. It writes.
You can ask any writer to write such a thing, using examples, and they could write similar things.
The real problem is that this will get deployed to the internet- there will be people reading HN today that are writing SEO optimised websites, with thousands of articles on a topic, that will just spring out of GPT4's writing, fully formed.
GPT can find the topics, pick a domain name from available ones, churn out the articles. It can all go into templates, with different AI generated graphic design supporting it. Ad supported churned out rubbish.
The writing style can change site to site, long form, short form, informed by current events, whatever. GPT would happily provide the prompts for this variety changing the style based on the topic and target audience.
It seems inevitable to me that the web WILL be full of such sites, and worse, they'll fill the comment sections on "social" sites too.
But? Banning AI isn't the answer, if for no other reason than it wouldn't work.
It's easy that something like this happens again. chatGPT confidently listes hallucinated sources->media rushes to publish the scoop -> now you have real "sources" for future reference
The fix will most likely have something to do with requiring that citations use primary sources, not just any article on the internet. In then end state, Wikipedia will be much better for it.
Ask it about why World War II started, or how to make a cake, or where to go for dinner, or anything else, and it gives you a confident, reasonable answer. A lot of the answers are simply whatever it's already seen, mashed up. You can think of it as a search. But actually it doesn't think about what it's saying, it's stringing words together to make you think it's smart.
So then when it makes up something, it will sound to you, the reader who always sees it answer in perfect English with a decent answer, like it found an article about this professor in its dataset and is merely summarizing it.
And people are trying to replace doctors with LLMs. It's like "ER" meets "Who's Line?"
This isn't a problem restricted to ChatGPT, there are humans who display this trait too. This might be appealing at a superficial level, but if you start believing speakers with this trait it's a slippery slope. A very slippery slope.
I'm trying really hard to avoid Godwin's law, so let me suggest that Elizabeth Holmes could be one example of this.
It is capable of spewing excellent bullshit(TM) at incredible rates, and always with the greatest expression of confidence and good grammar.
Occasionally, when in the 'middle of the road' of it's training set, it can provide useful output. So, it's better than the broken clock that is correct twice every day.
But, wander off the bulk of the training, and it is far worse than nothing; it is dangerously misleading unless you are very skeptical and knowledgeable in your field.
(I have tested it in several fields with the same results, interesting and helpful, yet hallucinating facts everywhere.)
Getting LLMs to know the difference between a good fact and a hallucinated mashup of plausible BS is looking like almost as large a problem to solve as making the LLMs in the first place.
[0] Bullshit is defined as written or spoken without regard to the truth, only what sounds good in the context. It's not a deliberate lie, just a salad of truth and falsehood, delivered without doubt or stress of lying.
Prompt: What did Glenn Beck do in 1990?
Answer: I'm sorry, but I do not have access to information on Glenn Beck's activities in 1990. Glenn Beck is an American television personality, radio host, author, and political commentator who has been active in various roles since the 1990s. However, without specific details or context, it's impossible for me to determine what he did in 1990. If you have any additional information or context, please let me know and I'll try my best to assist you.
I followed up with a much more specific version of that question, and it tripped the "this might violate the content policy" warning, but it did give the correct answer (that it was a hoax).