People constantly recommend Bard to me, but it returns false and/or misleading info almost every time
alwayshasbeen.jpg
There have been articles about how "data is the new oil" for a couple of decades now, with the first reference I could find being from British mathematician Clive Humby in 2006 [0]. The fact that it rings even more true in the age of LLMs is simply just another transformation of the fundamental data underneath.
I am specifically referring to the phrase I quoted, not some more abstract sentiment.
Bard's probably just a middle man here.
The response: "I'm a text-based AI, and that is outside of my capabilities."
For me it returns a seemingly accurate answer [1], albeit missing his involvement with Twitter/X. But LLMs are intrinsically stochastic, so YMMV.
Another interesting line of inquiry (potentially revealing some biases) is to ask it whether someone is a supervillain. For certain people it will rule it out entirely, and for others it will tend to entertain the possibility by outlining reasons why they might be a supervillain, and adding something like "it is impossible to say definitively whether he is a supervillain" at the end.
I really think they need to train on the wider dataset, then fine tune with some training on a machine specific dataset, then the model can reference data sources rather than have them baked in.
A lot of the general purposeness but also sometimes says weird things and makes specific references is pretty much down to this I reckon...it's trained on globs of human data from people in all walks of life with every kind of opinion there is so it doesn't really result in a clean model.
If you ask the same questions to ChatGPT you tend to get much more refined answers.
Ironing out is definitely the part where they're tweaking the model after the fact, but I wonder if we don't still need to separate language from culture.
It could help really, since we want a model that can speak a language, then apply a local culture on top. There's already been all sorts of issues arise with the current way of doing it, the Internet is very America/English centric and therefore most models are the same.
Personally, I struggle with anything even slightly technical from all of the current LLM's. You really have to know enough about the topic to detect BS when you see it... which is a significant problem for those using it as a learning tool.
I don't know the name for the effect, but it's similar to when you listen/watch the news. When the news is about a topic you know an awful lot about, it's plainly obvious how wrong they are. Yet... when you know little about the topic, you just trust what you hear even though they're as likely to be wrong about that topic as well.
The problem is people (myself included) try to use GPT as a guided research/learning tool, but it's filled with constant BS. When you don't know much about the topic, you're not going to understand what is BS and what is not.
Obviously they need to fix that for realistic usage, but I use it as a studying technique. Usually when I ask it to give me some detailed information about stuff that I know a bit about, it will get some details about it wrong. Then I will argue with it until it admits that it was mistaken.
Why is this useful? Because it gets "just close enough to right" that it can be an excellent study technique. It forces me to think about why it's wrong, how to explain why it's wrong, and how to utilize research papers to get a better understanding.
Like...it always has been?
There's the problem... and it defeats the entire purpose of using a tool like GPT.
I just ignore how confident ChatGPT sounds.
> OpenAI’s technologies had the lowest rate, around 3 percent. Systems from Meta, which owns Facebook and Instagram, hovered around 5 percent. The Claude 2 system offered by Anthropic, an OpenAI rival also based in San Francisco, topped 8 percent. A Google system, Palm chat, had the highest rate at 27 percent.
https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...