Hacker News Reader

Comment by duskwuff | Hacker News Reader

There's a hell of a lot more involved in hosting 400+ TB of data [1] than "renting a computer".

[1]: https://discourse.nixos.org/t/nixos-foundations-financial-su...

400TB of disk and a 100Gbe NIC in a single server is easily affordable hardware for a hobbyist, let alone any kind of organization

yeah, there will be maintenance and setup work, but we've (web software engineers) developed a weirdly extreme fear of self hosting, even in cases where we're being killed on transfer costs

Errr... What do you mean by "easily affordable"? A quick look on Amazon says that I'd be paying something like €18.80/TB, so buying 400TB would be €7,520 - far out of the range of the majority of hobbyists. Plus a 100GBE NIC would be another €800 or so, plus you'll need all the other ancillary things like a case and a processor and RAM and cabling and whatever the monthly cost to host it somewhere, unless your ISP gives you a 100Gbps connection (which I'm sure will be extortionately expensive if they do). End result is the hardware costs alone approaching €10,000 - significantly more if you want any sort of redundancy or backups (or are you planning to run the whole thing in RAID0 YOLO mode?), plus probably low-hundreds per month, not including the time investment needed to maintain it.

I mean, I spent about half that just on the computers in my apartment and I only have a 56Gbe LAN

I shouldn't say "easily affordable" like anyone can afford it, but rather something that you can do with off the shelf retail components (within the budget of a hobbyist software engineer, anyway)

it's cheap compared to hobbies like driving a sports car, at least!

You seem to have seriously skewed notions of what’s considered a hobbyist

Why? You can fit that in a single server. None of the data is unrecoverable. Most is binary cache for built packages and the ISOs can simply be rebuilt from what’s in git. They’re paying ridiculous amounts of money to serve easily rebuilt data.

The old binaries are not necessarily simple to rebuild. Projects move / go offline. Tags get moved. Binaries for the impure systems may not be public anymore. You can't easily rebuild everything from that cache.

Depending on how bad & frequent you want outages to be and whether you want the whole thing to go down for scheduled maintenance, it can get quite a bit more complicated. A pair of servers with some level of RAID is probably the minimum to achieve any amount of availability better than a server in someone's closet. Then you need a cold backup, on top of that, no matter how many live servers you have.

I've also had much better luck with connectivity, transfer speeds, and latency, for clients in some regions, with AWS than with some other hosts (e.g. Digital Ocean). It seems peering agreement quality really matters. Not sure how the big physical server hosts fare on that front vs. AWS (which is basically the gold standard, as far as I can tell) and it's hard to find such info without trying and seeing what happens. This matters if you're trying to serve a large audience over much of the Earth. The easy single-server solution might well see some clients lost 95% of their transfer speed from your service, or not be able to connect at all, compared with AWS.

Reply on news.ycombinator.com