HNHacker News
TopNewBestAskShowJobs

jonah-archive

2,118 karma · joined November 29, 2016

Infrastructure at Internet Archive: archive.org/donate

jonah AT archive DOT org

https://bsky.app/profile/jonahedwards.com

-

submissionscomments
jonah-archive··on Internet Archive Infrastructure
Thank you for this! It was definitely geared towards an internal audience but it makes me very happy to know that it was enjoyed and appreciated more broadly.

I am going to get a transcript done and up soon as well -- I just gave the talk on Friday so haven't had time to do so yet.

jonah-archive··on Let's Bring Spacer GIFs Back
I recommend that everyone go check out Sian Bahram’s 2016 talk on computing while blind. At around 6:30 he specifically addresses how inaccessible HN is to a visually impaired user: https://www.youtube.com/watch?v=G1r55efei5c
jonah-archive··on Iconic Print, “The Great Wave Off Kanagawa,” Recreated with 50k Lego Bricks
If you're curious about the process I would highly recommend David Bull's pamphlet and series of youtube videos detailing the extensive process of recreating the print in 2014/2015: https://mokuhankan.com/catalogue/0293.php
jonah-archive··on Transmits AM radio on computers without radio transmitting hardware
Reminds me of Tempest For Eliza, which does this using a CRT (now a little less practical): http://www.erikyyy.de/tempest/
jonah-archive··on Flash Animations Live Forever at the Internet Archive
Yes! See krapp's comment below -- we're using the Ruffle flash emulator targeted to WASM
jonah-archive··on Flash Animations Live Forever at the Internet Archive
Really excited about this, and so happy we were able to roll it out before Flash's official deprecation at the end of the year. If you have old SWFs you'd like to share, these are the directions for uploading them so that they'll be emulated in-browser: https://bluemaxima.org/flashpoint/datahub/Uploading_SWFs_for...
jonah-archive··on File_id.biz – A Mysterious Web CLI?
Hah, nice find —- this is mine, it was briefly a weird little puzzle based on Shakespeare (“I am not what I am” -> Iago), though I haven’t used the http endpoint for much in awhile and it doesn’t work anymore. Now that I work at the Internet Archive, I should make it an interface to some of our BBS archives!
jonah-archive··on When Math Gets Impossibly Hard
You can definitely do this! Check out Karplus's "Analog Simulation" for examples with conductive fluids and resistance networks: https://archive.org/details/analogsimulation0000karp
jonah-archive··on Details on the Ham Radio Equipment Being Used on the ISS
Exactly right on the repeater -- you communicate with the ISS on a VHF uplink and it rebroadcasts your communication on a UHF frequency (Earth stations typically run in a single band with a small offset). It's hard to talk to the ISS because it's moving so fast that there's a Doppler shift in the frequencies, but it's definitely possible with cheap gear (I've picked it up many times, but never tried talking through it). People aren't using it to communicate per se, more just to make brief long-distance (DX) contact with other amateur radio users.

There are also occasional opportunities to speak directly with people on the ISS (or more commonly, receive telemetry or slow-scan TV images).

jonah-archive··on Mysterious Inheritance: A new biography of J.B.S. Haldane
This is a great essay. If the topic interests you, D'Arcy Wentworth Thompson's classic work On Growth And Form is also a worthwhile read: https://archive.org/details/ongrowthform00thom
jonah-archive··on Activists rally to save Internet Archive as lawsuit threatens site
Oh, Jason's numbers are fine. He's not the one asserting that you can buy storage for <1¢/GB (and he correctly notes that the Wayback takes up >20PB).
jonah-archive··on Activists rally to save Internet Archive as lawsuit threatens site
No, that's for a single raw copy (it could go lower based on implementation -- the sweet spot on $/bit pricing is around 8TB/disk right now, but that would actually be more expensive for us in total cost because of the increased infra/space/power necessary to run them). We could probably get a relatively trivial 20-30% savings on space for Wayback contents via compression, maybe more with work (various projects are underway to do this), but much of the rest of the contents are difficult to compress, or already compressed (music, imagery for books, video, software archives, etc). We have also historically been very reluctant to deduplicate heavily, though we are experimenting with it for certain types of content -- one principle of operation is that as an archive of last resort, we're unable to have a true deaccession plan as some other archives have. A compromise we make is that our hard drives are "landfill-ready" -- that is, the contents of a drive (assuming you can read the filesystem) are inherently meaningful, content is housed with its metadata, and so forth. This produces some unusual restrictions on how far we can take compression and certain types of bitwise redundancy.
jonah-archive··on Activists rally to save Internet Archive as lawsuit threatens site
Disclaimer: I run the infrastructure/ops team at the Internet Archive.

Unfortunately none of those numbers are really even close to correct (the discussion is always fun, but the folks in r/datahoarder are often not correctly informed. textfiles has more patience for it than I do). It would probably cost around 1.5M in drives, even at reasonable current enterprise volume pricing, to back up the 60+ PB of unique data in the Internet Archive (plus, as someone does note in that thread, the cost of running them -- even if it were a static backup to cold disks, you still need chassis to run them in for the backup process, space and infra for them, electricity, people, &c). I don't know offhand how much space the contents of the Wayback currently take up, but it's definitely an order of magnitude more than that number as well.

jonah-archive··on Thank you for helping us increase our bandwidth
> Also, I'd trust these guys not to mishandle .org

As much as I like the idea of running a TLD, if anyone gives me a TLD, I'm gonna put an MX record on it and be jonah@org. That's a promise.

EDIT: To be fair, I would probably be overwhelmed with the usual sense of responsibility and not do this. But, the temptation

jonah-archive··on Thank you for helping us increase our bandwidth
Happy to chat in much greater detail over email if you like (in my profile) -- transit specifically is not generally our limiting cost (though of course in the context we operate in, every penny counts -- e.g. we run our own DCs, ambiently cooled, single-feed grid power with minimal backing, etc). We generally run very close to the edge, capacity-wise, in order to get the most out of what we have -- in this case, demand moved very quickly and we had to work a hardware deployment to catch up.
jonah-archive··on Thank you for helping us increase our bandwidth
It's harder when you don't make money, and everyone's not downloading the same things.
jonah-archive··on Thank you for helping us increase our bandwidth
Sibling comments are exactly correct -- we're a library. We don't track or record the activity of our patrons (see e.g. https://archive.org/services/docs/api/views.html#footnote-wh... ), and since most CDNs cannot offer the same guarantee, we can't use them.
jonah-archive··on SSH hacks – a little sanity for remote workers
Unfortunately it seems to be a very limited command set (tried out a few other things with no luck):

    ssh> ?
    Commands:
          -L[bind_address:]port:host:hostport    Request local     forward
          -R[bind_address:]port:host:hostport    Request remote forward
          -D[bind_address:]port                  Request dynamic forward
          -KL[bind_address:]port                 Cancel local forward
          -KR[bind_address:]port                 Cancel remote forward
          -KD[bind_address:]port                 Cancel dynamic forward

Time to read some source, I suppose.
jonah-archive··on SSH hacks – a little sanity for remote workers
Does anyone know of a modification that will e.g. dump hostname on a ~? -- so many times I'm in a few layers deep and likely to miscount.
jonah-archive··on The Internet Archive and Jason Scott are saving our weird Internet history
Thank you for your donations! We do our absolute best to put them to the most effective use we can.
jonah-archive··on Verizon extends deadline for Yahoo Groups shutdown to January 31, 2020
Sorry, I was trying to get the date/etc into the field. Maybe someone can suggest a better title.

Given that this was just announced, we don't really know what's going to happen, but it does seem like they're going to disable access on the 14th as expected, and provide their broken tool for downloads after that (until the latter deadline). All the more reason to download as much as we can as quickly as we can, and continue putting pressure on Verizon.

jonah-archive··on Sand Theft
See in particular this fascinating recent article regarding GPS jamming and sand theft in the port of Shanghai: https://www.technologyreview.com/s/614689/ghost-ships-crop-c...
jonah-archive··on Verizon extends deadline for Yahoo Groups shutdown to January 31, 2020
Hi all, Archive Team is still working on backing up as much of Yahoo Groups as they're able to, as discussed in this previous post: https://news.ycombinator.com/item?id=21737696

Info about the archival process: https://www.archiveteam.org/index.php?title=Yahoo!_Groups

Details about downloading groups you're a member of: https://docs.google.com/document/d/1qNCAgMN-Q9koFK1EzEFek2k4...

Tooling to do this from the CLI: https://github.com/ArchiveTeam/yahoo-group-archiver

If you want to help the effort, run yahoogroups-joiner to help the Archive Team join as many public groups as possible: https://github.com/davidferguson/yahoogroups-joiner

Run the Archive Team Warrior if you have available compute/storage: https://archiveteam.org/index.php?title=ArchiveTeam_Warrior

(Disclaimer: I don't work directly with Archive Team, but I work for the Internet Archive, which is where most of this data will end up living hopefully forever. We're also in the midst of our annual fund drive -- https://archive.org/donate/ if you're able and interested in helping out monetarily.)

jonah-archive··on Archivists Are Trying to Make Sure LibGen Never Goes Down
Some of the disks in the market over the last few years were not well-labeled in terms of revealing their SMR internals, and use larger media caches to disguise write performance issues (at least, until they don't). For example, the Seagate STGY8000400 is an external 8TB SMR drive. But the industry as a whole is moving to host-managed SMR, so hopefully that specific issue with external disks will soon go away.
jonah-archive··on Archivists Are Trying to Make Sure LibGen Never Goes Down
> at no greater or lesser annual failure rate than the expensive enterprise hard drives.

I've read these reports as well, but I can say that it's not my experience (we've gone through a few rounds of shucking at the Internet Archive, for economy and in one case necessity after the 2011 Thailand floods pinched the supply chain). Our raw failure rates on shucked drives are significantly higher, and the drives themselves are typically non-performant for high-throughput workloads (often being SMR disks/etc, though hopefully the move away from drive-managed SMR will finally kill that product category off).

jonah-archive··on Save .org
A couple additional things: EFF blog post on the topic: https://www.eff.org/deeplinks/2019/11/nonprofit-community-st...

Here's the initial letter being sent from EFF & others to ICANN and the Internet Society: https://www.eff.org/document/coalition-letter-sale-public-in...

(Disclaimer: I work for the Internet Archive, and we are one of the initial signatories to this letter.)

jonah-archive··on Nonprofit Community Stands Together to Protect .ORG
Here's the initial letter being sent from EFF & others to ICANN and the Internet Society: https://www.eff.org/document/coalition-letter-sale-public-in...

And here's the site for the campaign: https://savedotorg.org

(Disclaimer: I work for the Internet Archive, and we are one of the initial signatories to this letter.)

jonah-archive··on Stab a Book, the Book Won't Die
Moby Dick can be read aloud in 24 hours (well, at least experimentally closer to 26 -- we just did it! it's super fun: https://maritime.org/events/mobydick/ ). I've read it (silently) in 12. However, I don't agree with the assertion that reading a novel requires that rapid pace to fully enjoy it, though I've personally found that I'm unable to resume reading a novel on my Kindle the way I am with a physical text.
jonah-archive··on The Wayback Machine: Fighting Digital Extinction in New Ways
If anyone's interested in attending our party next Wednesday but the cost is presenting a difficulty, shoot me an email (in my profile) and I'll send you a ticket at no cost.
jonah-archive··on More MS-DOS Games Playable at the Internet Archive
Oh wow, hadn't seen that one -- that was in the aftermath of the 2011 Thailand floods that seriously impacted hard drive availability, there was almost nothing in the supply channel so we had to shuck external drives.

Sadly there's not really a great one-stop page for this kind of info -- https://archive.org/web/petabox.php exists but is sadly outdated (I'll see about rewriting it sometime). We do public tours on Friday afternoons of the main office/datacenter (and our big annual event is coming up next Wednesday if you happen to be in the Bay Area: https://www.eventbrite.com/e/world-night-market-the-internet... )

← PreviousPage 5 of 6Next →