Hacker News Reader

Comment by eastdakota | Hacker News Reader

DNS traffic, no. Random garbage traffic misdirected to 1.1.1.1, APNIC is studying.

Does all this garbage traffic affect the performance of Cloudflare's servers? There must be some cost (performance and $$$) to filter this traffic. Was that a consideration when deciding whether to use 1.1.1.1 instead of some other IP address? :)

Vendors like Cloudflare have usually really easy to use and cheap measures to drop traffic otherwise they would not be able to provide DDOS protection. Usually they have BGP or other means to propagate blacklisted IP ranges to peers meaning that not even their peers will route the garbage towards them. This is how you can survive DDOS that is bigger than your pipe while serving legit traffic (coming from different ranges than the ones you blacklisted).

No. We have a lot of capacity. A lot.

For ordinary singlehomed users who don't get the "a lot". As an example cloudflare has 40Gbps of capacity to the SIX in Seattle. I would guess that they also have direct, at minimum, 10Gbps PNI peering sessions with other huge ISPs in the Pacific Northwest which never see the SIX fabric. So probably add another 20 individual 10GbE circuits at bare minimum to that 40 figure. All of which helps spread the traffic load out rather than shoving it all down a few pipes.

https://peeringdb.com/net/4224

that's just the public stuff!

For huge entities like this and other top 5 CDNs it makes me wonder how many full time staff positions are dedicated to buying rack space, power and crossconnect in major colo facilities worldwide. How many contract law experts, telecom real estate analysts, etc. That before you even get into things like experts in Japanese contract law. Just for rack, power and facilities at layer one of the osi model before any networking happens.

I don’t know the exact number, but you’re off by about an order of magnitude roughly. Cloudflare peering is in the terabits/sec range globally.

At many non profit IXes the interface size to the fabric is public data and published by both the IX and on peeringdb. Such as at the SIX. They have not yet upgraded to 1x100GbE.

Another regional example, they have 20Gbps to the VANIX.

What is opaque is the size and scale of their PNI peering, which parties generally don't share. For example in a mid sized city where Comcast is the cable monopoly they almost certainly use a 100GbE interface direct to Comcast for just that isp.

Yes the scale is terabits globally. But it is highly decentralized.

> But it is highly decentralized

Isn’t that the entire point of a CDN, to have decentralized POPs scattered globally?

Yes, they may max out at 100gb per public IX in most cases, but they still have lots of 100gb peers all over the globe.

Yes, it's exactly the point. What I was saying is I am not off by an order of magnitude, I know exactly how big they are. Two ASes I do engineering work for peer directly with cloudflare.

> I am not off by an order of magnitude, I know exactly how big they are.

Your post implied a total of 200-400gb, the real number is 10x that (otherwise known as “order of magnitude”). I’m not disputing your knowledge or experience, but the post as written has issues.

no, my post was quite specifically about a single IX point in one geographical location, and was accurate. You confused me talking about interface sizes and counts at one IX point with cloudflare as a whole.

What difference does it how much they have as peering in one specific geo? Are you going to posit then that that is their capacity in that geo? I hope not, as Cloudflare has their own backbone and could easily and efficiently exchange traffic outside the geo depending on peering agreements.

Cloudflare is anycast announcing the space from something like 30 to 50 unique POPs worldwide, so the volume of shit traffic is significantly decentralized. It's not like 40 Gbps is hitting one location.

They scaled up recently and now have ~150 POPs. They specialised on DDOS defense from the start so traffic volume isn't an issue for them. Everything below 100gbit/s globally will probably be well within normal fluctuations. But I guess they talk to ASNs that send a lot of garbage and ask them to investigate on their side.

Sure, but they could also study DNS traffic if they wanted to. Or at least, with Cloudflare's cooperation.

If you want privacy, you never do DNS queries from an ISP-assigned IP address. Tor exits do DNS queries on behalf of clients. Decent VPN services also handle DNS queries for clients.

They don’t get raw DNS traffic. Ever.

But that's just a Cloudflare policy, isn't it?

Or are you arguing that even Cloudflare couldn't get raw DNS traffic?

That’s our policy and we’ve hired outside auditors to ensure we’re honoring it. If you have suggestions of what else we can do to prove we’re a company of our word, LMK.

Oh, I didn't realize that you're a CloudFlare cofounder.

I don't mean to question CloudFlare's integrity.

It's just that, for claims about privacy, I'd rather depend on more than trusting any one party.

Great. Use 1.1.1.1 and 8.8.8.8. You don’t depend on one party now?

Yes, thanks.

Even if you just use 1.1.1.1 you know you have 2 parties, cloudflare and their auditor.

There is that argument. But both are likely vulnerable to coercion from some adversaries. And of course, that's always a risk.

Will these auditors guarantee that you won't wake up one morning after a troublesome sleep and start monitoring this to protect us against Nazis? Because that might be good to ensure we believe your word.

I have a problem with these comments because they're basically a false dichotomy. No, CloudFlare can't ensure they play fair. Can your ISP? Who can?

Because these sorts of comments read to me as "yeah, you're the best right now, but are you perfect? No.". Nobody claimed perfection, and there's value in being the best.

Precisely. There's no perfect. OK, so I use IVPN. I've known a principal for many years. I write stuff for them. And I trust them to protect my privacy. More than any other VPN service. I also trust a few others, almost as much. And way more than I trust my ISP.

But I never depend on any one of them. I use nested chains. That's my no means perfect, because routes are relatively static, unlike Tor with its frequently changing circuits. But the basic idea is the same. Compromise would depend on collusion, perhaps forced, of multiple parties. Or some serious traffic analysis.

Here, there's CloudFlare, its auditors, and perhaps Google. So maybe there is distributed trust in that. But still, I'm happier to put more independent parties between me and them. Tor, or at least a nested VPN chain.

Cloudfares CEO was guaranteeing that they have an external auditors to ensure the company keeps its word. I'm asking whether the same auditors can ensure that the CEO himself won't decide one morning to go against this word wrt this topic. In the same way that a ToS might state "we will never sell your data to third parties" and a CEO can't break that promise, what guarantee beyond "some external people are looking at what we do" and "oh, just trust me" do we actually have? Im asking if the auditors will also guarantee the CEOs word.

This is relevant as the CEO has previously woken up one morning after a troublesome sleep and it has been argued, gone against his word (for good and anti nazi reasons). Many argue that he was entirely within his right to do so, and he was! So in this case, would he be entirely within his right to start monitoring all that data. As he asks, paraphrasing: "what more can I do to ensure my word is good."

I'd bet that if they one day drop that promise from their page they'll quickly hit the HN front page. If you don't trust, just monitor their policy page for diffs.

Will they at least get processed information about the DNS traffic (like how much of it is normal recursive queries versus broken garbage which happened to go to port 53)?

Reply on news.ycombinator.com