Hacker News Reader

Comment by rich_sasha | Hacker News Reader

Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient.

The flip side was, calligraphy was sufficient evidence for both his education to whoever hired him, and for a recipient of a document, of its official nature. Calligraphy itself or course didn't make him efficient or smart or fair.

That's long gone of course, but we had similar heuristics. I am reminded of the Reddit story about an AI-generated mushroom atlas that had factual errors and lead to someone getting poisoned. We can no longer assume that a book is legit simply because it looks legit. The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true.

LLMs are fantastic at breaking our heuristics as to what is and isn't legit, but not as good at being right.

View on HN

> We can no longer assume that a book is legit simply because it looks legit.

The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet".

I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation. Maybe LLM education will shift people back toward skepticism - unlikely, but I can hope.

One of the issues today is the volume of content produced, and that journalism and professional writing is dying. LLMs produce large amounts of "good enough" quality to make a profit.

In the 90s we could reasonably trust that that the major news sites and corporate websites was true, while random forums required a bit more critical reading. Today even formerly trusted sites may be using LLMs to generate content along with automatic translations.

I wouldn't necessarily put the blame on LLMs, this just make it easier. The trolls and spammers was always there, now they just have a more powerful tool. The commercial sites now have a tool they don't understand, which they apply liberally, because it reduces cost, or their staff use it, to get out of work, keep up with deadlines or to cover up incompetence. So, not the fault of the LLMs, but their use is worsening existing trends.

> Today even formerly trusted sites may be using LLMs to generate content along with automatic translations.

Yep - or they're commingling promotional content with their journalism, a la Forbes / CNN / CNET / About.com / etc. There's still quality content online but it's getting harder to find under the tidal wave of garbage.

> I was speaking to a librarian the other day who teaches incoming freshman how to use LLMs. What was shocking to me is that the librarian said a majority of the kids trust what the computer says by default. Not just LLMs, but generally what they read. That's such a huge shift from my generation.

I think that previous generations were not any different. For most people, trusting is the default mode and you need to learn to distrust a source. I know many people who still have not learned that about the internet in general. These are often older people. They believe insane things just because there exists a nicely looking website claiming that thing.

Not sure of the context here is for "previous generation" but I've been around since early in the transition from university/military network to public network, and the reality was the internet just wasn't that big, and it was primarily made up of people who looked, acted and valued the same things.

Now it's not even the website of undetermined providence that is believed; positions are established based on just the headline, shared 2nd or 3rd hand!

> The problem is that this has been an issue for a long time. My first interactions with the internet in the 90s came along with the warning "don't automatically trust what you read on the internet".

I received the same warnings, actually it was more like “don’t trust everything you read on the internet”, but it quickly became apparent that the last three words were redundant, and could have been rephrased more accurately as “don’t trust everything you read and hear and see”.

Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable. If you can’t change what someone sees, then you can probably change how they see it.

I feel like there was also a brief window where "many amateur eyes in public" trumped "private experts"; wikipedia, open source software, etc. This doesn't seem the case in a hyper-partisan and bifurcated society where there is little trust.

Believe half of what you see and none of what you hear

True!

> Our parents and teachers were living with their own fallacious assumptions and we just didn’t know it at the time, but most information is very pliable.

Indeed. When I was 14-18 in the UK, the opinion pieces in the news were derogatorily describing "media studies" as "Mickey Mouse studies".

In retrospect such courses were teaching critical analysis of media sources, in much the same way that my history GCSE went into the content of historical media and explored how both primary and secondary sources each had the potential to be either accurate or biased.

Even now, even here, I see people treat the media itself as pure and only the people being reported upon as capable of wrongdoing — e.g. insisting that climate scientists in the 70s generally expected an imminent ice age, because that's what the newspapers were saying.

>That's long gone of course, but we had similar heuristics.

To quote someone about this:

>>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life.

A book looking legit, a paper being peer reviewed, an expert saying something, none of those things were _ever_ good heuristics. It's just that it was the done thing. Now we have to face the fact that our heuristics are obviously broken and we have to start thinking about every topic.

To quote someone else about this:

>>Most people would rather die than think.

Which explains neatly the politics of the last 10 years.

> To quote someone about this: >>All that is solid melts into air, all that is holy is profaned, and man is at last compelled to face with sober senses his real conditions of life.

So, same as it ever was?

Smoke, nothing but smoke. [That’s what the Quester says.] There’s nothing to anything—it’s all smoke. What’s there to show for a lifetime of work, a lifetime of working your fingers to the bone? One generation goes its way, the next one arrives, but nothing changes—it’s business as usual for old planet earth. The sun comes up and the sun goes down, then does it again, and again—the same old round. The wind blows south, the wind blows north. Around and around and around it blows, blowing this way, then that—the whirling, erratic wind. All the rivers flow into the sea, but the sea never fills up. The rivers keep flowing to the same old place, and then start all over and do it again. Everything’s boring, utterly boring— no one can find any meaning in it. Boring to the eye, boring to the ear. What was will be again, what happened will happen again. There’s nothing new on this earth. Year after year it’s the same old thing. Does someone call out, “Hey, this is new”? Don’t get excited—it’s the same old story. Nobody remembers what happened yesterday. And the things that will happen tomorrow? Nobody’ll remember them either. Don’t count on being remembered.

c. 450BC

One is a complaint that everything is constantly changing, the other that nothing ever changes. I don't think you could misunderstand what either is trying to say harder if you tried.

"everything is constantly changing!" is the thing that never changes.

You sound like a poorly trained gpt2 model.

Culd be my KJV upbringing talking, but personally I think there's an informative quality to calling it "vanity" over smoke.

And there's more reasons not to simply compare the modern challenges of image and media with the ancient grappling with impermanence. Tech may only truly change the human condition rarely, but it frequently magnifies some aspect of it, sometimes so much that the quantitative change becomes a qualitative one.

And in this case, what we're talking about isn't just impermanence and mortality and meaning as the preacher/quester is. We'd be lucky if it's business as usual for old planet earth, but we've managed to magnify our ability to impact our environment with tech to the point where winds, rivers, seas, and other things may well change drastically. And as for "smoke", it's one thing if we're dust in the wind, but when we're dust we can trust, that enables continuity and cooperation. There's always been reasons for distrust, but with media scale, the liabilities are magnified, and now we've automated some of them.

The realities of human nature that are the seeds of the human condition are old. But some of the technical and social machinery we have made to magnify things is new, and we can and will see new problems.

'הבל (hevel)' has the primary sense of vapor, or mist -- a transient thing, not a meaningless or purposeless one.

Heuristics don't have to be perfect to be useful so long as they improve the efficacy of our attentions. Once that breaks down society must follow because thinking about every topic is intractable.

The mushroom thing is almost certainly true. There's tons of trash AI generated foraging books being published to Amazon. Atomic Shrimp has a video on it.

> Some great grand ancestor of mine was a civil servant, a great achievement given his peasant background. The single skill that enabled it was the knowledge of calligraphy. He went to school and wrote nicely and that was sufficient.

Similar story! Family lore has it that he was from a farming family of modest means, but he was hired to write insurance policies because of his beautiful handwriting, and this was a big step up in the world.

> The story of course is from reddit, so probably untrue, but it doesn't matter - it totally could be true.

What?! Someone just made up something and then got mad at it. This is specially weird when you even acknowledge its a made up story. If we start evaluating new things like this nothing will ever progress.

Reply on news.ycombinator.com