HNHacker News
TopNewBestAskShowJobs

jonah-archive

2,118 karma · joined November 29, 2016

Infrastructure at Internet Archive: archive.org/donate

jonah AT archive DOT org

https://bsky.app/profile/jonahedwards.com

-

submissionscomments
jonah-archive··on They're made out of meat (1991)
Another of Terry Bisson's short stories, "Bears Discover Fire", is an absolute gem, and free to read online: https://www.lightspeedmagazine.com/fiction/bears-discover-fi...
jonah-archive··on Three Arrows Capital reportedly facing insolvency
One of my all-time favorite examples of an alternate communication channel is when, their URL banned from in-game chat, gold sellers spelled it out in corpses on a hillside: https://www.reddit.com/r/wow/comments/2y2vb3/back_in_wrath_g...
jonah-archive··on Internet Archive opens Vancouver headquarters, meeting space for the tech world
Our numbers as of the end of 2021: https://archive.org/web/petabox.php

    4 data centers, 745 nodes, 28,000 spinning disks
    Wayback Machine: 57 PetaBytes
    Books/Music/Video Collections: 42 PetaBytes
    Unique data: 99 PetaBytes
    Total used storage: 212 PetaBytes
jonah-archive··on Asemic Writing
A cool example of this is Allison Parrish's book generator project, a system of writing-like imagery produced by a generative adversarial networks that your brain really, really wants to think are real letters: https://github.com/aparrish/word-gan-book-generator

Aleator Press (specializing in computer-generated books) also recently published a letterpress book of pseudo-poetry from it: https://www.aleator.press/releases/wendit-tnce-inf

jonah-archive··on Ultra compact GAN ATX power supply delivers up to 250 Watts
The SGI O2 was built like this (power & all bus connections were off an edge connector), some good pics of the mobo at: https://bukosek.si/hardware/collection/sgi-o2.html#pics
jonah-archive··on Internet Archive joins opposition to the “SMART Copyright Act”
Thank you! We do our absolute best to do as much as we can as best as we can with your donated funds. It's really appreciated.
jonah-archive··on Ask HN: Teach me something new
The big bang was extremely hot; the universe now is quite cold (microwave background). It was a continuous process of cooling, implying that there was an interval where the entire universe was comfortable (there are at least a few papers on this, e.g. https://arxiv.org/abs/1312.0613 , but I like it more just as a strange thing to thing about).
jonah-archive··on Ask HN: Teach me something new
and for the curious -- cw stands for Continuous Wave, a radio term for signaling that's turned on and off (like Morse code -- as opposed to modulated like AM, FM, etc).
jonah-archive··on Slack’s Incident on 2-22-22
RIB is a common term in networking for "Routing Information Base" (being the set of all routes which could be chosen to be installed in the routing table (or FIB -- "Forwarding") by the control plane. I don't know that this is the actual etymology but it's not implausible.
jonah-archive··on Meta quickly detects silent data corruptions at scale
uh oh, I recognize this one. Love to have a file corrupt after months at rest with no access logged and no mtime changes because a file on a neighboring track needed rewriting (SMR, of course).
jonah-archive··on US Senate votes unanimously to make daylight savings time permanent
Ooof, I remember that. At the time I was writing shipping logistics optimization software for LTL shipping, and many clients were using ancient warehouse inventory systems (lots of data uploads over ftp, etc) that couldn't easily be modified to account for the time change, uh, change. Very painful.
jonah-archive··on Playdate Software Development Kit is now available for free to all
Announcement tweet: https://twitter.com/playdate/status/1498427655175159822
jonah-archive··on Incident: Air Asia A320 near Kuching on Feb 10th 2022, snake on the plane
Same -- went to the midnight (iirc they just moved it ahead to 10pm, since unlike LotR/etc no one cared) show at the Cinerama in Seattle and it was packed. People brought pool noodles to wave around, the whole crowd was hissing -- I've actually never rewatched it because I assume nothing could compare to the experience of watching it in that setting.
jonah-archive··on Albrecht Dürer: Self-publicist, tourist, entrepreneur, Renaissance artist
If you find this interesting, I cannot recommend Philip Hoare's "Albert and the Whale" (mentioned in the article as well) highly enough -- the best work yet by a great writer: https://www.simonandschuster.com/books/Albert-and-the-Whale/...
jonah-archive··on Why pine nuts are expensive [video]
You can harvest your own (Pinus monophylla) pine nuts in Great Basin National Park: https://www.nps.gov/grba/planyourvisit/pinenutgathering.htm -- I highly recommend it, it's a great park and you'll really get a sense for the labor involved (and some pine nuts). If you're in the Bay Area the drive out across Route 50 (the Loneliest Road In America) is beautiful as well.
jonah-archive··on AT&T’s Abandoned Microwave Tower Network (2017)
Spotted one up on Steens Mountain not so long ago: https://www.google.com/maps/@42.4584052,-118.6942825,155m/da...
jonah-archive··on Apple’s app review prevents developer from submitting fix to game for the blind
Here's a talk by Sina Bahram from five years ago -- at around 6:30 he specifically addresses how inaccessible HN is to a visually impaired user. Nothing that he mentioned has been addressed: https://www.youtube.com/watch?v=G1r55efei5c
jonah-archive··on Did AS8003 just disappear?
I mean... yes. And then there's the real world where your parent company has bought a dozen small companies over the years who have things spread across all of the 1918 ranges and you're setting up VPNs and static NAT and RIP and IS-IS paths and... you just do something easier. It's not a _good_ idea, but it's definitely a thing that happens (fortunately, for me at least, it's been a good long time since I've had to deal with that brand of network badness).
jonah-archive··on Link rot and content drift are endemic to the web
Thank you! We're doing what we can with what have where we are, or maybe just what we must because we can, depending on your preferred aphorism :)
jonah-archive··on Link rot and content drift are endemic to the web
We've got a pretty solid diff view now, notes at https://blog.archive.org/2019/10/18/the-wayback-machine-figh... and an example at https://web.archive.org/web/diff/20170118202526/201701200403...

Re slowness, we're able to do a lot with a little, but there's always room for improvement. If you're interested in some of the specific infrastructural challenges, I did a presentation in February:

https://archive.org/details/jonah-edwards-presentation

and my colleague did a fantastic presentation detailing some of the internal workings of the Wayback Machine just last week:

https://archive.org/details/bridget-bell-presentation

jonah-archive··on Don't Piss Off Bradley, the Parts Seller Keeping Atari Machines Alive
Oh my gosh, thank you for this -- I have an old CGP-115 and found the replacement gears awhile back, but hadn't found a source of pens other than Bradley and had a couple unsuccessful attempts to purchase them from him. The mini pen plotter shall live again!
jonah-archive··on The Graphics Gremlin – An Open Source Retro ISA Video Card
Typically when they do, they're built in to server chassis as a remote management/LOM tool. A few such add-on cards have been made over the years, though -- the PC Weasel 2000 was quite a nice card back in the day: https://web.archive.org/web/20110406024209/http://www.realwe...

Others like the Enidan J1 were specifically targeted toward people building clusters out of commodity hardware: https://web.archive.org/web/20010511215150/http://www.enidan...

jonah-archive··on Light Pollution Map
This is a bit tangential, but should be a pretty good aurora over North America tomorrow (Sat Apr 24) night -- visible on the horizon as far south as Salem, OR to Annapolis, MD: https://www.gi.alaska.edu/monitors/aurora-forecast

(forecast is for the UTC day, so the prior overnight for NA)

jonah-archive··on Show HN: I made a tool to communicate data using the PC speaker
at one point the iPod bootloader was extracted by playing it aloud: http://www.ipodlinux.org/stories/piezo/
jonah-archive··on Ask HN: What tech job would let me get away with the least real work possible?
Congratulations -- I left VC-funded tech for a nonprofit a few years back and it's the best thing that's ever happened to me (or I guess, that I did). It's certainly not perfect but working in an environment with people who believe in what they're doing and genuinely want to support each other has made a massive difference for my attitude as regards work in a general sense.
jonah-archive··on We may soon have city-spanning 900 MHz mesh networks
!! do you have the PDFs/etc that the Wayback didn't capture? If so, perhaps I can host them on the Internet Archive? Particularly interested in the Garage Gateway -- I've never seen the code and I have an otherwise working Ethernet Radio & pair of poletops I'd love to get properly running.

E-mail's in my profile -

jonah-archive··on We may soon have city-spanning 900 MHz mesh networks
Some are -- they pop up on local government auction sites (and even ebay) from time to time. The modems themselves are actually still quite useful, you can direct-dial between two of them using AT-style commands over the serial port, or they can be operated in a packet-radio style mode using tooling like strip: http://man.netbsd.org/NetBSD-6.0/strip.4

(STRIP was actually in the linux kernel, though I assume long gone or defunct -- the only things I still have talking via Ricochet are 2.4-era)

I have a couple of the lightpole radios (they listen for the modems on 900MHz and speak amongst themselves on 2.4GHz) and scored a single tower radio (talks to the lightpole radios and has an ethernet downlink) and have gotten them to speak with each other, but haven't had time to figure out the route mapping steps -- IIRC the deal is that the packet routing path information is stored upstream and delivered to the downstream radios. No idea if anyone ever decoded that format.

I messed around with this a lot in the immediate post-Ricochet era in between flashing silver WaveLAN cards to gold... fun times. It seems like it would be entirely possible now to run a simulated node using SDR.

jonah-archive··on Internet Archive Infrastructure
Thank you, so glad you enjoyed it!

We try to keep a regular upgrade cycle (to hold to our tight budget), typically with a tick-tock of adding new hardware (expanding within our existing footprint) and cycling out old hardware and drives as they reach the far end of the works/doesn't work spectrum. We have a local partner who takes care of some disposal for us, but we also have no shortage of physical storage space, so we will also accumulate (sometime intentionally -- nearly our entire "red box" deployment -- see "previous version": https://archive.org/web/petabox.php -- is packed into a shipping container. We don't like to throw things away!).

For a hypervisor, we use Ganeti (running over KVM). Because our fleet is so heterogenous we need to be able to control a lot of VM parameters in order to efficiently pack our computational resources, and Ganeti is kind of in a sweet spot for us in terms of providing a lot more tooling than a bunch of virsh scripts, and being much smaller than systems like OpenStack geared towards large, homogenous deployments).

jonah-archive··on Internet Archive Infrastructure
Exactly correct. I suspect that in the not-too-distant future we will need to move to multi-disk filesystem-level clustering on the storage nodes for some of the reasons laid out in the talk but it's not at all unlikely that we retain the "disconnected mirror" abstraction for redundancy.

Additionally, as I alluded to in the video, we regularly end up going down into the details and when this happens being able to closely examine and understand a disk's contents as written directly at the LBA using fibmaps and similar tooling is invaluable. I have personally been involved in the discovery of multiple possible-data-loss hard drive firmware bugs, and our catalog system is very paranoid about integrity checking. Modern hard drives are computers unto themselves and layering additional complexity on top of that (particularly complexity which might obscure or silently correct errors in the underlying datastream) is something I approach very carefully.

jonah-archive··on Internet Archive Infrastructure
Exactly this. There are some logistical complexities (e.g. some of our bandwidth is funded by the E-Rate Universal Service Program for libraries which runs on a July-June fiscal year and so rapid upgrades on that front aren't possible), but by and large egress bandwidth isn't our primary challenge. Intersite links as I noted in the video are the current big one, and that can and does involve occasional time-consuming construction -- but honestly over the past year, a combination of total blowout of my usual capacity planning (including equipment budgets) plus the logistical complexities of lockdown have resulted in slowness to upgrade as fast as we'd like to.
← PreviousPage 4 of 6Next →