Hacker News Reader

Comment by cperciva | Hacker News Reader

60 Gbps is about 1/3 of the traffic served by a single Netflix CDN node.

It's harder when you don't make money, and everyone's not downloading the same things.

I understand money is tight for a non-profit, but why are bandwidth costs a gating factor?

Both your upstreams, Cogent and Hurricane Electric, offer 100G ports at fiveish grand per month in carrier neutral DCs. Given that your budget is in the millions, an outlay of this magnitude doesn't seem wildly out of the question.

If you can explain what the problems are in getting more bandwidth, I'd be more than happy to see what I can do to help.

Please let me know if you'd prefer to discuss the matter over email.

Happy to chat in much greater detail over email if you like (in my profile) -- transit specifically is not generally our limiting cost (though of course in the context we operate in, every penny counts -- e.g. we run our own DCs, ambiently cooled, single-feed grid power with minimal backing, etc). We generally run very close to the edge, capacity-wise, in order to get the most out of what we have -- in this case, demand moved very quickly and we had to work a hardware deployment to catch up.

I'm sure I'm not the only one who'd be interested in reading more of this. Is there a reason your conversation can't proceed on HackerNews?

Or at least update us on the conclusion :)

You can follow the thread on the NANOG list:

https://mailman.nanog.org/pipermail/nanog/2020-May/107720.ht...

Email sent.

You are comparing to one of the largest bandwidth using services world-wide. Of course archive.org will lose out.

I should have added: 60 Gbit/s is a lot for a donation run service like archive.org. Not sure if there are donation run services on the internet with more traffic.

> single Netflix CDN node.

Looks like they max out at 100 Gbps, or as low as 40 Gbps, depending on appliance model and link aggregation configuration. No argument either way, just thought it was cool info.

https://openconnect.zendesk.com/hc/en-us/articles/3600345383...

That's out of date. Here's a presentation from last year where they talk about >190 Gbps: https://people.freebsd.org/~gallatin/talks/euro2019.pdf

The video accompanying this slide deck is also a great watch: https://www.youtube.com/watch?v=8NSzkYSX5nY

I'd say the "bytes" being transferred are very different.

Netflix has a (relatively) small number of large content pieces, and a CDN node would hold the hottest items.

IA has 430 billion individual pieces of data, on average much smaller than a piece of Netflix content.

So each "byte" transferred from IA is "more work" to produce, and less likely to come from some kind of cache.

does archive.org use cdn or is it all served from one location?

Source?

Reply on news.ycombinator.com