Lights candles to summon Jason Scott to ingest it into the Internet Archive
Lights candles to summon Jason Scott to ingest it into the Internet Archive
And he has responded. https://lars.ingebrigtsen.no/2016/07/28/the-end-of-gmane/com...
I was not aware of this. Do you have a link?
edit: self clarifying my own question. Not sure what the specifics for hosting an AWS Public data set are though.
Big Query
>A public dataset is any dataset that is stored in BigQuery and made available to the general public. This page lists a special group of public datasets that Google BigQuery hosts for you to access and integrate into your applications. Google pays for the storage of these data sets and provides public access to the data via BigQuery. You pay only for the queries that you perform on the data (the first 1 TB per month is free, subject to query pricing details).
[0] https://cloud.google.com/bigquery/public-data/
AWS
> Public Data Sets on AWS provides a centralized repository of public data sets that can be seamlessly integrated into AWS cloud-based applications. AWS is hosting the public data sets at no charge for the community, and like all AWS services, users pay only for the compute and storage they use for their own applications. Learn more about Public Data Sets on AWS and visit the Public Data Sets forum.
The professor I was working with told us about how he lost all his data once with a similar startup attempt. Shortly after I was let go from the startup I pulled all the data via the web service I wrote, luckily before they shut it down.
TL;DR I'd trust an entity like the Internet Archive way over commercial entities, even those that offer free storage for open projects. Sure the Internet Archive could up and go away, but I still trust their motives and their commitment to not let that happen.
The difference would be that if AWS itself wanted to stop providing public archive hosting, they'd just up and do that. Whereas if the Internet Archive was hosting a public archive on AWS, and AWS cut off support for public archives, the Internet Archive would then work to migrate the dataset off AWS.
Non-profits have incentives to do (whatever their charter says they do.) Corporations just have an incentive to make money. If you want a certain "preference function" to carry on into the future without self-modifying into something unrecognizable, you probably want to instantiate that preference-function in the form of a non-profit.
But no matter what, the best possible solution would probably be to move the archive to the Internet Archive, and have that same archive continue to ingest email->nntp and provide access to it.
I get that people fear spammers etc, but at this point, I don't see a way to avoid it if you want a usable web archive.