Hacker News Reader

Comment by est31 | Hacker News Reader

60 Gbit/s of continuous traffic is a lot. If I'm reading the graphs right, Wikipedia "only" has 13.4 Gbit/s of outbound traffic [1]. Of course still well below the single-digit Tbit/s traffic of large internet exchanges [2] [3], but still unexpectedly large.

[1]: adding up the outbound numbers for each datacenter: 1.888 + 8.003 + 810 + 1.958 + 807: https://grafana.wikimedia.org/d/000000605/datacenter-global-...

[2]: https://en.wikipedia.org/wiki/List_of_Internet_exchange_poin...

[3]: Not 100% correct as thanks to Corona, ix.br now has passed 10 Tbit/s peaks. https://ix.br/noticia/releases/ix-br-reaches-mark-of-10-tb-s...

View on HN

60 Gbps is about 1/3 of the traffic served by a single Netflix CDN node.

Source?

You are comparing to one of the largest bandwidth using services world-wide. Of course archive.org will lose out.

I should have added: 60 Gbit/s is a lot for a donation run service like archive.org. Not sure if there are donation run services on the internet with more traffic.

> single Netflix CDN node.

Looks like they max out at 100 Gbps, or as low as 40 Gbps, depending on appliance model and link aggregation configuration. No argument either way, just thought it was cool info.

https://openconnect.zendesk.com/hc/en-us/articles/3600345383...

That's out of date. Here's a presentation from last year where they talk about >190 Gbps: https://people.freebsd.org/~gallatin/talks/euro2019.pdf

The video accompanying this slide deck is also a great watch: https://www.youtube.com/watch?v=8NSzkYSX5nY

I'd say the "bytes" being transferred are very different.

Netflix has a (relatively) small number of large content pieces, and a CDN node would hold the hottest items.

IA has 430 billion individual pieces of data, on average much smaller than a piece of Netflix content.

So each "byte" transferred from IA is "more work" to produce, and less likely to come from some kind of cache.

It's harder when you don't make money, and everyone's not downloading the same things.

I understand money is tight for a non-profit, but why are bandwidth costs a gating factor?

Both your upstreams, Cogent and Hurricane Electric, offer 100G ports at fiveish grand per month in carrier neutral DCs. Given that your budget is in the millions, an outlay of this magnitude doesn't seem wildly out of the question.

If you can explain what the problems are in getting more bandwidth, I'd be more than happy to see what I can do to help.

Please let me know if you'd prefer to discuss the matter over email.

Happy to chat in much greater detail over email if you like (in my profile) -- transit specifically is not generally our limiting cost (though of course in the context we operate in, every penny counts -- e.g. we run our own DCs, ambiently cooled, single-feed grid power with minimal backing, etc). We generally run very close to the edge, capacity-wise, in order to get the most out of what we have -- in this case, demand moved very quickly and we had to work a hardware deployment to catch up.

Email sent.

I'm sure I'm not the only one who'd be interested in reading more of this. Is there a reason your conversation can't proceed on HackerNews?

Or at least update us on the conclusion :)

You can follow the thread on the NANOG list:

https://mailman.nanog.org/pipermail/nanog/2020-May/107720.ht...

does archive.org use cdn or is it all served from one location?

Wikipedia is mostly text with some images. Internet archive offers every hung from websites and text to movies and games, which obviously are much larger and don't compress as nicely either.

I wonder what the comparison between served data is for IA vs Wikipedia - anecdotally, I feel like most of the data for Wikipedia is text based (plus some number of images), whereas IA includes (as mentioned in the article) audio, video, etc.

Reply on news.ycombinator.com