Isn't all of this true for a human-used internet as well?
When an LLM is the lens through which you view that text, you often don't have the context of where it came from, and the LLM is often really bad at filtering trustworthiness based on context when retrieving information to answer a query with.
I wouldn't go to a random site with a sketchy URL to get the customer service number for my xyz. I'd go to their official domain. Thus successfully navigating the treacherous waters scammers attempt to muddy.
In principle AI Agents are superhuman at codeforces but tons of these vibe coded apps have ridiculously stupid performance problems.
The problem is a utilization of expertise thing, it can tell you with incredible detail about pros and cons and caveats to a problem if you ask about it. Truly superhumanly intelligent.
Calibration when it comes to actual decision making takes an incredible reduction in capability after that. Maybe it has something to do with the sycophancy or something but its like someone with exactly zero opinions (and therefore, taste).
My experience is that you can brainstorm with it on "Do A or B?" and it will produce a coinflip answer A, then if you continue and ask about a specific concern "what about this aspect of B?", it immediately 180s and flips back to B being way better than A.
Acting on this expertise when decision making points arise naturally for itself is where everything really nosedives. It's just gonna do whatever.
So when you are doing this "financial advice please" questioning, it is entirely hitting these weaknesses squarely in the face.
Gullible is an appropriate colloquialism here. As in they don’t check countervailing narratives naturally and fall into a recency sink on last context. Long walk context construction is hard, time consuming and most Importantly significantly more expensive.
The incentive will be to maintain a subscriber base.
Insanely optimistic view to take. I know a handful of people who make this effort in my personal life, including me, and we tend to be treated as paranoid for caring.
You have an entire lifetime of experience and the wisdom of countless generations to inform you of the context with which you should process new information. You were specifically trained for years to be able to do this somewhat effectively. Doing the same for an LLM may or may not be possible. They contain much of that experience in some form, but are not well equipped to make use of it iteratively. It is incredibly expensive to attempt even a simple and ineffective version of it with our current methods.
If the LLM can achieve even 90% accuracy, it becomes more of a hindrance than a help. Do you make use of the function and trust it successfully understood the provenance of new information? If so, ten percent of the time you will be misled, which could be disastrous across a population. If you don't simply trust it, then do you painstakingly check the provenance of everything it tells you? Of course not, you already don't do that with current search engines and blogs and such. Yet falling somewhere in between would be so much worse than simply not having the LLM attempt to discern trustworthiness from providence in the first place.
But all of that is opinion and I may be incorrect. Perhaps someone will create some ingenious way of imparting wisdom and discernment to LLMs next year and this will all be trivial.
With training data, however, no such annotation is possible.
During training? Not a chance.
The padlock on browsers means nothing to most humans. Permission popups are just there to be immediately accepted so that one can start scrolling reels sooner rather than later.
I don't think the average human has a particularly great sense of discernment either
I’m not sure how exactly that generalizes to LLM behavior, but to you as a human cultivating discernment, I imagine it implies that anything Internet-mediated (and unvetted by you) is less reliable than it used to be.
The real lesson is this: trust in the source is the most important thing for 99.99% of what you read. You will sometimes read about facts that you can actually verify, but those are mostly useless in the grand scheme - except as a sanity check on the source. For everything else, you have to ultimately trust the source.
Now sure, sometimes you can get lucky and detect certain internal inconsistencies that prove that something can't be true. And to some extent, reading the same information from multiple sources you don't normally trust too much can be a slight signal that it might be true. But these are generally in the minority.
Again I am not ignoring reality. But for society to work and for us to live our daily lives with any semblance of sanity, we can't be expected to be fully informed on everything we see, touch, consume, etc. We have to be able to make reasonable assumptions about safety and accuracy or we can't function. And the time it takes to vet stuff is often a privilege (or you can brute force pay your way to safety).
I do not rigorously audit everything I read on wikipedia. I assume a certain baseline of accuracy up to a point, with of course exceptions. Do you pull up 3-4 high quality sources for everything you read or claim? Are you that academic about it?
Why is it so unconscionable to hold tech corporations accountable? We know (maybe I should say knew at this point?) how to enforce fraud and consumer protection. I fail to see how this shouldn't fall under commercial liability that's really all that different than run of the mill false advertising or product defects.
If we fail to do this we subsidize the operators and externalize the cost of fraud onto the general public.
I fail to see how any of this is even vaguely comparable to, say, stopping the spread of fake news. How would you write laws that allow the government to stop the spread of fake news, but not block the release of true but highly inconvenient news? For example, should Big Tech be allowed to spread news about the stolen US election, as the President of the United States calls it? How about the Genocide in Gaza, that the United Nations recognizes but that the United States calls fake news and antisemitic propaganda? And even if you have a clear opinion on which of these is fake news and which isn't, do you trust this administration and all future ones to have the same opinion?
And if I'm misunderstanding that you are talking about regulating the spread of fake news, please suggest what specific regulations you're thinking of.
Also, I didn't say you need to access high quality sources for everything, I again kind of said the opposite: you choose who to trust for what, and that's that. There is nothing more, and has never been anything more, that you can do. Sometimes people and sources you trust will still lie to you, or be wrong, and you'll know wrong information - this is also unavoidable.
Of course, the more important some information is to you, the more effort you should actually spend trying to verify with multiple trusted sources. Some weird fact about cats you saw in a TikTok? Fine to just believe it. Trying to figure out who to vote for, or if the elections were stolen? Probably worth checking a few more sources. Read some definition for a word on Wikipedia, or some old historical event? Probably fine. Reading about an even slightly controversial topic? Check the actual sources, don't trust the snapshot you happen to see*.
Of course, the way LLMs work is antithetical to this, as information provided directly by the LLM is not coming from any human source you can decide if you should trust or not. So you shouldn't trust it for anything important, unless you can verify. But for information or disinformation on the web, nothing has changed, except maybe the quantities.
*for a silly example: the exact definition of the word "electrocution" is a slightly controversial topic - whether electrocution refers to any (significant) injury caused by electricity, or only death. Pedants soemwaht regularly edit the article on Wikipedia to remove the "or serious injury" part of the definition, despite it carrying a link to the OED that includes this mention.
Now, the bigger problem happens if some of the trusted institutions and people go bad, for various reasons. If the NYT got bought out by Alex Jones, it might start mixing in lots of slop - but this is also a pre-existing problem.
Yes, but if you read this "semantic" info directly from a site, you can judge the site and its potential biases, and decide for yourself "this sells Louis Vuitton, of course they will praise their handbags". Or to pick a more extreme example that nobody would ignore, if you somehow land on an extremist website spewing racist crap, your alarms will surely trigger by anything they say.
But through an LLM this gets whitewashed and presented as authoritative and "neutral", when it isn't. You might know not to trust the LLM either (if you're paying attention; we all have our unconscious biases) but I bet you most people will be caught off guard.
Billions have been spent on it.
Try searching Bing sometime. First result for Firefox is a malware clone site (you’ll have to scroll past the malware ad to see it).
(I am not including AI results in this).