Hacker News Reader

Even if we still had the Library of Alexandria, it may have shed zero light on the actual lives of citizens. Archiving content on the internet means capturing thousands of individual level perspectives and experiences. We don't know what will end up being important to historians 50 or 100 years from now. I would bet there are dozens if not hundreds of historians that would give anything for a record of their favorite time period that contains even a fraction of the amount of content today's archive efforts are storing.

It's also not horrendously expensive - we are getting better and better at storage as well data analysis techniques, so stuff that seems useless today may be useful 50 years from now and cost less to store than it does now. The key thing again being that we can't benefit from hindsight.

Even graffiti can give insight into a time period, even if that insight is that that time period had an unusually high number of graffiti artists.

YES! It's like preserving ecological diversity. It's a store for later learning. Verizon is working in cold hard capitalism, and you can bet your lunch that they did NOT use Google Groups to hold their shared wisdom/history, and they would never let it be lost.

But many don't have the pockets for better systems, and so their earned knowledge lived on Google Groups. And when you think of all the people and groups that might have had needs to store their history, and what tools they might have used, what do you expect the skew of Yahoo Groups was. Certainly no Fortune 500 companies, but rather nonprofit and grassroots and all sorts of domains that are already getting the short end of the stick in our world :)

Heh *Yahoo Groups, that is

For example: World War two groups where many of the the members have passed away by now. There could be first hand accounts of history that has already been lost to time.

Could?

More like definitely.

What about people who don't want stupid comments they made online when they were 14 permanently indexed and searchable for all of time by the Archive Team? Yes, they may have posted to Yahoo! Groups back in 1999 when they didn't know better, but now it's 2019 and you have people digging up decades-old dirt on people to try and destroy their reputations and careers.

Given that search engines have zero ethics when it comes to removing embarrassing (but not illegal) content, sometimes the loss of information is a small blessing for some.

Yes, it's their fault, but I also don't think it's fair that something a child said at 14 should haunt them their entire professional careers, either.

The archives are not easily indexable by search engines, they're posted as multi-GB gzip-compressed WARC files.

But someone could hypothetically convert the WARC files back to static HTML and host them on the clear web.

Hypothetically, yes; but right now all this stuff is available on the clearnet and searchable. So obviously any potential harm of the present situation, is decreased. And, unless your argument is that we should delete all fora on the web because someone may have said something embarrassing on them, then I think you'd probably want to come down on the side of preservation.

Given that search engines have zero ethics when it comes to removing embarrassing (but not illegal) content,

Ethics are about codified sets of rules. Perhaps they're just following a set of rules that doesn't promote hiding things to make people feel better?

IA are extremely responsive in delisting content on request.

Email info@archive.org

Withhold wide-scale, anonymous access for a few decades maybe? (Though presumably there is a middle ground that doesn't involving leaving _everything_ inaccessible for a few decades.)

I'm pretty sure Yahoo isn't doing this to protect people from their old posts.

The stuff stored in the Yahoo groups is material from the beginning of the internet. When people explored what could be possible and how easy is was to connect globally. You have a valid point, but it's also one of these things in our generation that we have to live with. We explored and tried things. Only now we look back and see what those explorations of our younger selfes really are; sometimes funny, sometimes embarrassing. However, if you are cautious, you may be able to delete your stuff or at least make it anonymous by deleting that said account. If not, you have live with it. Those of all these people can now learn from it and can educate their kids in being careful with the internet. (Or at least this is what it should be)

The dogma, that "everything posted to the internet will stay on the internet" , may not be entirely true for this first generation, because now large parts are already gone. But I am certain that this will be very true for the current generation, because I really doubt that Facebook and others will ever freely delete large datasets of user content.

Not to mention that historians of the future will be able to sort and characterize massive amounts of data and draw conclusions that couldn't be made without that data.

For a time period where data is more valuable that oil, that the wealthiest companies are trying to grab every piece of data they can, and on a site where this is frequently discussed and many work for said companies, I find the question "why do archivists want to archive data?" a little silly. Date might not be useful to us now, but might be to future historians (though this is a similar argument made by that companies that do mass surveillance).

Step 1: We only need to archive the genuinely good content.

Step 2: It will take a long time to look through all this content and determine which parts deserve keeping.

Step 3: We will inevitably leave out something that someone else thinks is worth keeping anyway.

Step 4: Let's just archive everything.

It's basically that. Yes, when saving everything we'll save a lot of trash and utterly garbage, spam and all that shit... But the things we would be risking to lose if we didn't save everything, they are and will be so much important. To save what is really important, you have to save everything.

And actually, spam is quite interesting to some people. It certainly gives a flavour of what early-2000s internet was like, and what happens when spam filters aren't good.

One man’s public toilet block graffiti wall is another’s Library of Alexandria. Let the historians and journalists decide what’s important and the archivists take their best crack at saving it.

I write a lot of historical content and often the most useful stuff I find—for example, old flyers or ads from the 1950s or 1960s—would have been considered trash by someone at the time.

So an archivist’s job isn’t to make a judgment. It’s to protect the data as they see fit.

Toilet wall graffiti and such, preserved in Pompeii, is an important archaeological resource for the understanding actual daily life of Romans.

So yes, there are real, hardcore scientific papers about ancients "shitposting" each other down to "your mom" jokes. Because it shows us how people really lived.

> I'm genuinely curious from an ideological perspective, why archivists think all this material is worth saving?

It's easier to just save it all and let gawd sort it out.

You never know what some future person might find interesting. For example, my father took lots and lots of pictures, but they're all set in the living room and kitchen. No pictures of the rest of the house. I'm sure the thought of photographing other rooms simply never occurred to him as being interesting.

For another example, many people are interested in where/when/why certain words first appeared, like the origin of "OK". Massive archives of text that are searchable would help with this.

It is a lot of resources thrown at saving, not the equivalent of the Library of Alexandria, but the public toilet block graffiti wall.

Ask an antiquarian about the value of graffiti in the ruins of Pompeii and other archaeological sites sometime. The great historians of the day wrote about their contemporary culture, while the vandals and miscreants and lowlifes and commoners contributed to that culture. Having access to both sources gives us a much more complete picture.

You don't know what's worth saving at the time you save it.

Ha, ha! Well, there's some high quality material there too, but I take your point. In the right context, like "history from below," all kinds of material can be high quality!

To be clear, I wasn't comparing the fanfic authors and other Yahoo Groups contributors to vandals scribbling dicks all over Pompeii. Just saying that all other things being equal, future historians will prefer to have too much data to work with than too little.

By definition, we don't have the benefit of hindsight until it's too late.

See below. My main concern is early medical/biohacking groups that shared data, like medical tests, and engaged in extensive discussion/community driven research. Such groups go back to at least the late 1990's.

A main concern of the Archive Group (again, below) is art that was uploaded there.

I'm sure those are not the only two classes of examples. See for example the bird watching group in Delhi that has been collecting data for decades. (In the link of the OP.)

Great question! I'll take an amateur swing at a decent answer:

People doing important work (esp important work that is underfunded) don't have time to write/record their own histories. But that history can be instructive, to learn what worked and what didn't, and help future travellers do it better :)

And perhaps especially important: ppl engaging in these under-resourced efforts are often working in domains that capitalism is... less curious about, we'll just say. Otherwise, it would likely be able to be more highly documented, as incentive is there to preserve it.

Our ability to improve our present from better understanding our past is a supposed benefit of a digital world that accrues data -- we have records of things that in prior ages just flew by in conversation (for better or for worse). But efforts like this rob us all of that wisdom <3

And again, there is an asymmetry in who gets robbed. It is often the folks working in the commons, those doing invisible maintenance labour (nonprofits, grassroots, community), and generally just people doing work within the cracks of capitalism.

One never knows what may have value.

The graffiti on the toilet wall may well speak to the start of a trend, term, movement, or other event, for example.

Think longer timelines, broader scope than you personally may feel is relevant.

En mass, those questions have answers we individually are unlikely to fathom.

> The LoA held knowledge collated and collected by the best thinkers of the time.

... that had access to writing services and were wealthy enough to have their thoughts stored.

There could have been many odd voices out there that would've told us an entire different story. But these are unknown because they didn't have access.

Now we are in the era of (almost) universal access to storing our thoughts and we still don't listen to the everyone or mark them as uninteresting and not worthy.

> It is a lot of resources thrown at saving, not the equivalent of the Library of Alexandria, but the public toilet block graffiti wall.

We have that kind of graffiti from Pompeji. It's enormously more fascinating and insightful into regular people's lives than all the stuff about kings and battles people wrote about in the more official works.

When looking through all newspapers and magazines, the advertisements are often the most interesting bit. Especially since you can probably already read about the big events they wrote about on Wikipedia or history books.

Certain group contents are actually unique and valuable (see threads below), in which there could be a certain similarities to the LoA.

But most importantly, Groups is a corpus representing many segments of society during a period (starting 2001, with a peak of over 100 million users in 2008). It's a snapshot that embodies concerns, beliefs, morals, language... at several realms. This is more than LoA even. It can be used profusely by researchers and historians to study society for years to come. Or by AI to learn how and who we are/were...

> The LoA held knowledge collated and collected by the best thinkers of the time. The Internet is not that.

I am in awe of your flair for understatement.

We don't think it's necessary to preserve everything that's ever spoken verbally. We don't lament that everyday conversation is ephemeral.

People are conflating internet discussion content with written content because it's stored as text. Whereas the more legitimate comparison is to verbal communication.

> We don't lament that everyday conversation is ephemeral.

I imagine you're not a historian. Neither am I, but I cannot imagine that there is a historian out there who hasn't lamented the ephemerality of everyday conversation (and even of apparently more durable forms of communication).

>We don't lament that everyday conversation is ephemeral.

Linguists definitely do.

Historians would love to preserve spoken communication and there are many projects recording everyday conversations. There are even projects of recording the typical sounds of the environment in certain areas at a certain time. However, many forms of spoken everyday conversation fall under restrictive privacy laws, which poses strict limits to such preservation efforts.

The texts on the internet at a given time, on the other hand, are public and reflect the opinions and ways of living of a large number of people at that time. There is no doubt that these could be analysed in the future to give us historical insights in ways we cannot even conceive yet. (Think e.g. about getting them data mined and analysed by advanced A.I. to give new insights into the time period.)

The worth of the data is so obvious that it's really hard for me to understand why you and some other people don't think these are interesting data points for research on how we lived in, say, 200, 500, or even 10000 years from now. The data is not only interesting to historians, but also to economists, political scientists, and linguistics, btw.