It's only more terrifying when you cross it with the idea that around 50% of the population already can't separate real from fake on the internet
(We've all heard of children torturing animals, especially cats. I don't know why that happens, but planting ideas or making light of it doesn't seem like a great idea.)
Problem solved, children learn a better lesson.
I base my belief in what I see online on cross-checking, general trust-worthiness of sources I encounter, and my gut feeling for the difficulty of faking something, but there's nothing stopping a determined adversary from inserting things into my perceived reality.
A sufficiently advanced AI could be generating the internet on the fly just for me and I wouldn't be any the wiser.
See:
- "Adult Literacy in the United States (2019)" <https://nces.ed.gov/pubs2019/2019179/index.asp> HN: <https://news.ycombinator.com/item?id=29734146>
- "Skills Matter: Further Results from the Survey of Adult Skills" a 2016 OECD report based on research conducted in 2011--12 and 2014--15 involving 215,942 adults: <https://www.oecd-ilibrary.org/education/skills-matter_978926...> NNGroup (Jakob Nielsen) writeup, "The Distribution of Users’ Computer Skills: Worse Than You Think" <https://www.nngroup.com/articles/computer-skill-levels/>
For both, the high-ability group is a small set (~15% for overall literacy, 5--10% for technical literacy) at the population level.
Note that this is highly consistent across multiple countries and does not seem to indicate a particular lack of ability amongst American residents. (Though the US literacy study does show a marked decline especially of non-native English speakers near the Southern border.)
This is sometimes described as the Lake Wobegon Effect (or perhaps "Lake Wobegon Fallacy"), which is to say that you can't exit a general population cognitive limit by assuming "all the children are above average" (from the late NPR radio programme "A Prairie Home Companion").
We're going to have to address the coming, and/or present, highly-contaminated epistemic environment with the population and skills we have, not those we'd prefer, to borrow from another sentiment.
For the last decade or so I've become increasingly worried that the open Internet and broad-franchise democracy may simply be, inevitably, incompatible, no matter what we might wish were the case, and that we're guaranteed to lose one or the other (or, potentially, both).
There's one thing for people to harm themselves due to their mistakes, and another thing entirely for them to harm others because of their mistakes.
Moreover, people don't learn from their mistakes if someone else is the one that suffers the consequences
Power seeks power, and power exerts power. It does so through markets, through criminal organisations, through militaries (nationally-affiliated, mercenary, or any of a continuum of hybrid forms), and more.
Liberal democratic governments are at least in name and theory accountable to the governed, and even many less democratic forms still reflect the collective will of diverse groups within their societies.
The Max Weber dictum that's so often misquoted and mis-construed doesn't hold that government has a monopoly on force, but that it has the monopoly to the legitimate claim on same.
Weber, Max (1978). Roth, Guenther; Wittich, Claus (eds.). Economy and Society. Berkeley: U. California Press. p. 54.
<https://archive.org/details/economysociety00webe/page/54/mod...>
Yes, limit government, and ensure that it is lawful and legitimate (as opposed to capricious and corrupt). But don't eliminate it, as then all that remains is violence used without legitimacy or restraint.
It seems like you agree that there should be limits to the scope of government as well.
We might differ on where those limits lie, but maybe not.
My primary point is that these limits are what mitigate the damage misguided individuals can do to others.
It's a frequently-enough expressed trope that I tend to anticipate it and act preemptively.
Generalising the problem, what I see is the need to moderate power, and to balance power, in the sense of John Kenneth Galbraith's counterveiling power.
<https://en.wikipedia.org/wiki/Countervailing_power>
In the context of generative AI, the problem is of an unbalanced new power, that of generating text or speech that appears semantically meaningful, but whose purpose and intent are unclear, perhaps even to the parties operating the AI itself. More generally, however, within any communications channel in which social, political, economic, or other significant potential exists, you can be quite well assured that some entity, agent, or power, human, machine, or hybrid, will emerge to attempt to exploit that opportunity. I first saw this realised by my friend Woozle, after whom I've termed this "Woozle's Epistemic Paradox", first expressed on Google+, though now preserved in a Reddit post of mine:
Our present epistemic systems are undergoing kind of the same shock that the online community underwent when transitioning from BBSs and Usenet to the commercial web to social media.
We were used to a very high content-to-BS ratio because it took a certain amount of intelligence and intense domain-interest for people to be there in the first place -- and we've now transitioned to a situation where many people are there more or less accidentally and (the worst part), because of a high percentage of the population being present, there is now substantial power to be had by influencing the discussions that take place.
<https://old.reddit.com/r/dredmorbius/comments/5wg0hp/when_ep...>
Bad ideas and misinformation are plentiful. They have always been in different media environments, although the speed and fervor may change with time.
If some people think that this puts democracy at risk, the solutions are the same.
Firewalls and limitations to restrict the ability of others to impose their opinions on you. A Lynch Mob is a very democratic event.
The only defense against Lynch Mob Behavior, especially in times of high suggestibility, is maintaining limits and reinforcing norms for what is off limits.
Something I've become exceptionally aware of in the past decade is that virtually every previous major shift in communications or informational capabilities has come with enormous disruption.
Several of those occurrences were within the previous century. We've seen two or three in the present as well, so far.
The concept is at the core of Elizabeth Eisenstein's work, The Printing Press as an Agent of Change, though that of course touches on only one such revolution itself.
Previously: <https://news.ycombinator.com/item?id=34906482>
(To clarify, she's thinking along the lines of personally controlled, managed filters, like those from the near-future parts of Neal Stephenson's Fall, not some sort of centralized control.)
Essentially, nobody with power trusted the mob with a vote until they could write convincingly. Apply that to each minority as they gained the right.
The history of the Long 19th Century (1789--1914) strongly suggests this, and it's a theme I'd noted in other histories of the period, e.g., John R. Gillis, The Development of European Society, 1770-1870. Notably chapter 10, "Europe's Cultural Revolution", concerning literacy, freedom of the press, schooling, changing roles of intellectuals, and so on.
I've noted in earlier HN comments that the 19th century saw a tremendous increase in the productivity of the press and a corresponding fall in the cost of printed materials. Hand-operated wooden "wine press" printing might reach 120 impressions or so per hour. An electrically-powered rotary web offset-lithographic press could reach 1 million impressions per hour --- that's a four-order-of-magnitude increase. See: <https://news.ycombinator.com/item?id=21679745>
If you look at the history of Britain, there's a near-continuous devolution of power to an ever-growing and expansive set of the population. Initially singular monarchs, then a nobility (Magna Carta), then a growing bourgeois class (~15th -- 18th centuries or so), then the working class (19th century) and women, arguably jews (Disraeli). In the past fifty years, a growing, if not monotonic, inclusion of former colonial subjects and people, to the point that London's mayor and in fact some nobility are no longer exclusively white.
There are multiple factors at play, though literacy all but certainly plays some role.