ChatGPT is only "truthy", however. The algorithm will happily give you information that sounds true, but is actually wrong. And if it's wrong even 1% of time, you will still need some way to verify its claims.
Remember how people use to criticize the newspapers? They'd say they read an article on a subject they new something about and were shocked at how bad it was. Then they'd conclude that the rest of the news was just as bad. What wasn't acknowledged was the utility of having imperfect information.
One of the early popular examples was asking ChatGPT what the downsides are of generics in TypeScript, and it cited that they're not compatible with some browsers/runtimes. It's entirely believable, but entirely wrong, and actually quite hard to confirm either way with internet searches. It's the sort of thing people would readily believe and pass around, though.
Similarly, when I was trying to concatenate video in ffmpeg, it produced convincing-seeming command lines, almost none of which worked properly, but almost all of which used actual ffmpeg command line parameters with appropriate-seeming formats and names. I wasted significant time on debugging those before realising quite how wrong ChatGPT was.
That's an interesting take, and one I haven't heard before. My main squeamishness with bad journalism, and even worse, with chat gpt, is not truth-content, but rather sourcing.
If you know the source, you can account for its biases, or, your reader can do so. As such, well-sourced quotes of complete nonsense tend towards adding to the sum total of truth in the world, because it was true that somebody said it, and their perspective is part of what's interesting in whatever event you're discussing.
If you scrub the data of its sources (like journalists do routinely, and chatgpt does by necessity) important information is lost.
Any bias is everyone’s bias. Not saying it doesn’t happy at all today . But wow, scary
We will look at these things as devine correctness, and eventually forget their human construction entirely.
As long as ChatGPT assists me more than it harms me it is quite powerful simply due to the number of different things it can assist me with, and the raw scalability.
The web is chock full of incorrect stuff, both unintentional and deliberately deceptive.
It's more like "First one on the list? Well, that one's right -- unless it disagrees with my personal biases, then maybe I'll look for another one."
In short, I see a need for us all to inherently distrust anything single thing we read -- everything needs to be verified and checked out across multiple (hopefully diverse) sources of information before it can be considered "truth" in any sense. I do develop trust in certain sources over time, but meta-sources like an AI or a search engine don't get this treatment because their scope is too broad. As an example, I tend to trust folks like Fabien Sanglard[0], Eric Goldman[1], and Adam Langley[2].
Example: over on the Morrowind subreddit a few years ago, an artist had been painting landscapes of the game "Morrowind" (oil on canvas, IIRC), and when he went to print them as posters to sell to fans, the site he wanted to print them on warned him that printing his own paintings of vistas from Morrowind was copyright infringement (it's not). He came to the Morrowind subreddit for help because he'd been actively misled.
Sites like plagiarism.org[3] confidently say false things:
> But can words and ideas really be stolen?
> According to U.S. law, the answer is yes. The expression of original ideas is considered intellectual property and is protected by copyright laws, just like original inventions. Almost all forms of expression fall under copyright protection as long as they are recorded in some way (such as a book or a computer file).
This is confidently incorrect in a couple of different ways. It might seem like incompetence (and may be), but it's worth noting that plagiarism.org is actually run by TurnItIn, which profits from people not really understanding how copyright works.
Similar example (I tend to look into copyright, trademark, and patent issues a lot): I got into a discussion with another HNer about IP law, and they cited an article written by an actual lawyer[4] on a site called "The National Law Review" that's simply wrong about IP law to defend their position. It's tragic, because we've built this sort of cargo cult around "citation, please?" but then all-too-often don't keep our critical thinking skills engaged as we evaluate the source.
This is true all over. I've read HN daily for more than 10 years. Not only does every comment need scrutiny (even if stated confidently), but most stories linked on the front page do as well. It's not just blog posts or opinion pieces; some scientific studies that are posted get torn apart in the comments, to enough of a degree that, after cross-checking, I tend to think the study itself is deeply flawed.
And there are numerous stories about how Google's Quick Answers confidently give the wrong answer, as highlighted in 2017 by the WSJ[5].
I'm not saying accuracy doesn't matter, but I feel the "ChatGPT is confidently wrong" needs more context or comparative analysis around it before it becomes a compelling argument, since "confidently wrong" applies to search engines and humans as well. I haven't seen any in-depth studies on this, but would love to.
[0]: https://fabiensanglard.net/
[1]: https://blog.ericgoldman.org/
[2]: https://www.imperialviolet.org/
[3]: https://www.plagiarism.org/article/what-is-plagiarism
[4]: https://www.natlawreview.com/article/enforce-your-intellectu...
[5]: https://www.wsj.com/articles/googles-featured-answers-aim-to...
The problem with "citation, please" is it only works if the person you are engaging with is acting in good faith.
If I type in "Can intellectual property rights expire if not litigated?" That web site (national law review) doesn't appear in the top 10 results. So I'd question where did they get that link?
Not so long ago I got in a debate with someone here whether muscle exercises can lead to long term increases in testosterone levels.
The person I debated with provided a link to an abstract of a study that didn't look quite right. There was no link to the actual study.
Instead of engaging with them I confirmed that, within a minute or two, I was able to locate a high quality, free to read, medical literature review on the topic.
Maybe my ability to type keywords into a search engine is superior, but instead I came to the conclusion the other poster was half assedly looking for search results to "win the argument" rather than engaging due to intellectual curiosity.