Hacker News Reader

Comment by scruple | Hacker News Reader

How do you get the poison out of the water in the well? I understand the way this is supposed to work on the web, there are support channels to deal with this sort of thing. But if the deceit is baked into the model, what is your recourse?

View on HN

Same as always: Develop a healthy sense of skepticism, cultivate a stable and diverse network of sources whose credibility you understand, don’t trust what you read, and certainly don’t trust synthesis unless you know how it’s made?

Of course, and I'm really not trying to move the goal posts here, but I agree that that's what (most) humans are going to do. But we're rapidly moving in a direction of agent-to-agent flow and "they" seem to lack the degree of discernment the average human does. They certainly don't have the same incentive structure we do to detect fraud. They happily regurgitate and build off of lies because, to them, the lies are not lies, and they can't be. They're just a pile of geometrically close labels from the perspective of the model.

> "they" seem to lack the degree of discernment the average human does

I don't think the average human has a particularly great sense of discernment either

And what’s more, the average human’s counterparties are exquisitely tuned to exploit the gaps in that natural endowment. But the tools to improve discernment are the same as they’ve always been, just as pushups and crunches lead to physical fitness.

I’m not sure how exactly that generalizes to LLM behavior, but to you as a human cultivating discernment, I imagine it implies that anything Internet-mediated (and unvetted by you) is less reliable than it used to be.

Seems like the next push, then, will be to get agents and LLMs to start doing that discernment?

Obviously, we don't move in a direction of agent-to-agent flow because it's fundamentally flawed and worse than the status quo

So “caveat emptor”? I get that’s always part of reality but we shouldn’t exactly steer into that skid IMO. It just shifts the burden to the rest of us and does nothing to address the root cause

It is, and always has been, the only possibility. It's up to every individual to check for themselves what is true and what is not, and to decide who to trust for things that they can't check.

The real lesson is this: trust in the source is the most important thing for 99.99% of what you read. You will sometimes read about facts that you can actually verify, but those are mostly useless in the grand scheme - except as a sanity check on the source. For everything else, you have to ultimately trust the source.

Now sure, sometimes you can get lucky and detect certain internal inconsistencies that prove that something can't be true. And to some extent, reading the same information from multiple sources you don't normally trust too much can be a slight signal that it might be true. But these are generally in the minority.

So you test every sip of water you drink? Inspect every road before you drive down it? Check every bathroom product for harmful additives?

Again I am not ignoring reality. But for society to work and for us to live our daily lives with any semblance of sanity, we can't be expected to be fully informed on everything we see, touch, consume, etc. We have to be able to make reasonable assumptions about safety and accuracy or we can't function. And the time it takes to vet stuff is often a privilege (or you can brute force pay your way to safety).

I do not rigorously audit everything I read on wikipedia. I assume a certain baseline of accuracy up to a point, with of course exceptions. Do you pull up 3-4 high quality sources for everything you read or claim? Are you that academic about it?

I think the point is that for those you are trusting the regulators. For stuff you read online you are either trusting the source of the information or not. You should choose what information sources you trust carefully.

That doesn’t really change the calculus for me. We can regulate whatever we want.

A lot of people are throwing their hands in the air and suggesting that this is all just some sort of unavoidable consequence of physics, rather than something that is being pushed on us without any ability to confront it.

Regulating information, while it's fairly common at some levels (e.g. school manuals, one of the largest sources of legal information control, and one often abused for nationalistic lies in every country's history), is a very dangerous thing for governments to get into. Do you think the Trump government would be more likely to correct the NYT or Alex Jones if they had the ability to regulate news stories?

There are many ways to tackle regulation that don’t involve giving the keys to fickle administrations.

Tackle the regulation of “truth”?

What does the FDA do when someone gets caught selling a fake medication? Do we wrap truth in scare quotes and try to make the sugar pills a philosophical issue?

Why is it so unconscionable to hold tech corporations accountable? We know (maybe I should say knew at this point?) how to enforce fraud and consumer protection. I fail to see how this shouldn't fall under commercial liability that's really all that different than run of the mill false advertising or product defects.

If we fail to do this we subsidize the operators and externalize the cost of fraud onto the general public.

Fake medication at a gas station or do you mean on the internet?

The FDA has a very limited ability to regulate "truth". It has the single ability to regulate what standards something has to meet in order to be called "medication", and slightly more generally, in order to be advertisable with very specific health claims. But that's about it - you are absolutely free to sell snake oil as long as you don't call it medication and you don't, say, claim it specifically cures alopecia, but simply claim that it helps with hair growth. The gigantic health supplements and homeopathic and alternative remedies industry is testament to that.

I fail to see how any of this is even vaguely comparable to, say, stopping the spread of fake news. How would you write laws that allow the government to stop the spread of fake news, but not block the release of true but highly inconvenient news? For example, should Big Tech be allowed to spread news about the stolen US election, as the President of the United States calls it? How about the Genocide in Gaza, that the United Nations recognizes but that the United States calls fake news and antisemitic propaganda? And even if you have a clear opinion on which of these is fake news and which isn't, do you trust this administration and all future ones to have the same opinion?

And if I'm misunderstanding that you are talking about regulating the spread of fake news, please suggest what specific regulations you're thinking of.

My whole point is that you have to trust people, that this is the only thing you can rely on for almost every important thing you care about. For food, for example, I don't eat from sketchy restaurants, unless I get some personal recommendations from friends about it. That is, I only eat from places where I think I have some reason to trust that the food is sanitary.

Also, I didn't say you need to access high quality sources for everything, I again kind of said the opposite: you choose who to trust for what, and that's that. There is nothing more, and has never been anything more, that you can do. Sometimes people and sources you trust will still lie to you, or be wrong, and you'll know wrong information - this is also unavoidable.

Of course, the more important some information is to you, the more effort you should actually spend trying to verify with multiple trusted sources. Some weird fact about cats you saw in a TikTok? Fine to just believe it. Trying to figure out who to vote for, or if the elections were stolen? Probably worth checking a few more sources. Read some definition for a word on Wikipedia, or some old historical event? Probably fine. Reading about an even slightly controversial topic? Check the actual sources, don't trust the snapshot you happen to see*.

Of course, the way LLMs work is antithetical to this, as information provided directly by the LLM is not coming from any human source you can decide if you should trust or not. So you shouldn't trust it for anything important, unless you can verify. But for information or disinformation on the web, nothing has changed, except maybe the quantities.

*for a silly example: the exact definition of the word "electrocution" is a slightly controversial topic - whether electrocution refers to any (significant) injury caused by electricity, or only death. Pedants soemwaht regularly edit the article on Wikipedia to remove the "or serious injury" part of the definition, despite it carrying a link to the OED that includes this mention.

Once this provenance trail, that humans use to trace claims back to an accountable source, is dead and buried under automated sludge, "just pick people to trust" stops working as a viable strategy. How do you even tell who is actually talking?

Seriously we’ve got Reddit now straight up translating foreign language comments with AI and without stating they did it. That is a massive decision to make, and I feel like no one cares.

I don't think it's necessarily so. I view it more like garbage collection algorithms - the amount of garbage doesn't actually matter, you should only be looking at the roots of trust and down from there, and those will generally only lead you to more trustworthy content. The WSJ won't randomly link to fake TikTok accounts.

Now, the bigger problem happens if some of the trusted institutions and people go bad, for various reasons. If the NYT got bought out by Alex Jones, it might start mixing in lots of slop - but this is also a pre-existing problem.

Reply on news.ycombinator.com