I am going to get a transcript done and up soon as well -- I just gave the talk on Friday so haven't had time to do so yet.
2,118 karma · joined November 29, 2016
jonah AT archive DOT org
https://bsky.app/profile/jonahedwards.com
-
I am going to get a transcript done and up soon as well -- I just gave the talk on Friday so haven't had time to do so yet.
There are also occasional opportunities to speak directly with people on the ISS (or more commonly, receive telemetry or slow-scan TV images).
Unfortunately none of those numbers are really even close to correct (the discussion is always fun, but the folks in r/datahoarder are often not correctly informed. textfiles has more patience for it than I do). It would probably cost around 1.5M in drives, even at reasonable current enterprise volume pricing, to back up the 60+ PB of unique data in the Internet Archive (plus, as someone does note in that thread, the cost of running them -- even if it were a static backup to cold disks, you still need chassis to run them in for the backup process, space and infra for them, electricity, people, &c). I don't know offhand how much space the contents of the Wayback currently take up, but it's definitely an order of magnitude more than that number as well.
As much as I like the idea of running a TLD, if anyone gives me a TLD, I'm gonna put an MX record on it and be jonah@org. That's a promise.
EDIT: To be fair, I would probably be overwhelmed with the usual sense of responsibility and not do this. But, the temptation
ssh> ?
Commands:
-L[bind_address:]port:host:hostport Request local forward
-R[bind_address:]port:host:hostport Request remote forward
-D[bind_address:]port Request dynamic forward
-KL[bind_address:]port Cancel local forward
-KR[bind_address:]port Cancel remote forward
-KD[bind_address:]port Cancel dynamic forward
Time to read some source, I suppose.Given that this was just announced, we don't really know what's going to happen, but it does seem like they're going to disable access on the 14th as expected, and provide their broken tool for downloads after that (until the latter deadline). All the more reason to download as much as we can as quickly as we can, and continue putting pressure on Verizon.
Info about the archival process: https://www.archiveteam.org/index.php?title=Yahoo!_Groups
Details about downloading groups you're a member of: https://docs.google.com/document/d/1qNCAgMN-Q9koFK1EzEFek2k4...
Tooling to do this from the CLI: https://github.com/ArchiveTeam/yahoo-group-archiver
If you want to help the effort, run yahoogroups-joiner to help the Archive Team join as many public groups as possible: https://github.com/davidferguson/yahoogroups-joiner
Run the Archive Team Warrior if you have available compute/storage: https://archiveteam.org/index.php?title=ArchiveTeam_Warrior
(Disclaimer: I don't work directly with Archive Team, but I work for the Internet Archive, which is where most of this data will end up living hopefully forever. We're also in the midst of our annual fund drive -- https://archive.org/donate/ if you're able and interested in helping out monetarily.)
I've read these reports as well, but I can say that it's not my experience (we've gone through a few rounds of shucking at the Internet Archive, for economy and in one case necessity after the 2011 Thailand floods pinched the supply chain). Our raw failure rates on shucked drives are significantly higher, and the drives themselves are typically non-performant for high-throughput workloads (often being SMR disks/etc, though hopefully the move away from drive-managed SMR will finally kill that product category off).
Here's the initial letter being sent from EFF & others to ICANN and the Internet Society: https://www.eff.org/document/coalition-letter-sale-public-in...
(Disclaimer: I work for the Internet Archive, and we are one of the initial signatories to this letter.)
And here's the site for the campaign: https://savedotorg.org
(Disclaimer: I work for the Internet Archive, and we are one of the initial signatories to this letter.)
Sadly there's not really a great one-stop page for this kind of info -- https://archive.org/web/petabox.php exists but is sadly outdated (I'll see about rewriting it sometime). We do public tours on Friday afternoons of the main office/datacenter (and our big annual event is coming up next Wednesday if you happen to be in the Bay Area: https://www.eventbrite.com/e/world-night-market-the-internet... )