684 karma · joined January 18, 2017
The threat model I proposed is coherent for Cloudflare because they have done a lot of engineering to make it almost impossible to globally BGP hijack their IPs. This makes the multi-perspective validation actually help. Yes, other ISPs are much more vulnerable than Cloudflare, is there a point?
You are not saying DNSSEC doesn't serve a real purpose. You are saying it is annoying to implement and not widely deployed as such. That alone makes me believe your argument is a bit dishonest and I will abstain from additional discussion.
I am saying it is dishonest to discount the real security threat of not having DNSSEC.
1. Hijack the HTTP/HTTPS server. For some IP ranges, this is completely infeasible. For example, hijacking a CloudFlare HTTP/HTTPS range would be almost impossible theoretically based on technical details that I won't go through listing.
2. Hijack the DNS server. Because there's a complete apathy towards DNS server security (as you are showing) this attack is very frequently overlooked. Which is exactly why in the cited incident attackers were capable of hijacking Amazon Route53 with ease. *DNSSEC solves this.*
If either 1 or 2 work, you have yourself a successful hijack of the site. Both need to be secure for you to prevent this.
DNSSEC does greatly assist with this issue: It would have prevented the cited incident.
"For instance, in April 2018, a Russian provider announced a number of IP prefixes (groups of IP addresses) that actually belong to Route53 Amazon DNS servers."
By BGP hijacking Route53, attackers were not only able to redirect a website to different IPs, globally, but also generate SSL certificates for that website. They used this to steal $152,000 in cryptocurrency. (I know I know, "crypto", but this can happen to any site: banking, medical, infrastructure)
Also, before you say, RPKI doesn't solve this either, although a step in the right direction. DNSSEC is a step in the right direction as well.
[1] https://www.cloudflare.com/learning/security/glossary/bgp-hi...
I do not agree that a state-of-the-art high capacity deployment is as expensive as you think it is. If an organization pays MSRP on everything, has awful procurement with nonexistent negotiation, and multiple project failures, sure, maybe. In the real world though, we're not all working for the federal government (-:
That Duplex Dark Fiber with DWDM can run 4TBPS of capacity at 100GE (40x 100GE). Each 100GE transceiver costs $2-4K NRC dependent on manufacturer - $160K NRC for 40x. (There are higher densities as well, like 200/400/800GE, 100GE is just getting cheap.)
In AWS, utilizing 1x100GE will cost you >$1MM MRC. For significantly less than that, let's say absolutely worst-case 5K MRC + 200K NRC, you can get 40x100GE.
Now you have extra money for 4x redundancy, fancy routers, over-spec'd servers, world-class talent, and maybe a yacht if your heart desires.
AWS egress data transfer pricing is 100-1000x the price of equivalent IP Transit. (Yes, I am not exaggerating.) It is so significant that I've seen data transfer account for >90% of 7-figure AWS bills. Given enough spend, that alone is enough to make the business case for a migration.
Even the largest ISPs generally do not have enough spare capacity to just "block UDP packets," as the largest attacks are now measured in Tbps. There is also nothing stopping attackers from just sending TCP packets.
As consumer internet speeds grow, DDoS grows.
Bandwidth is cheap.
ISPs at scale do not use software routers. They use ASIC routers (Juniper/Arista/Cisco/etc.), for many reasons 1) features 2) capacity 3) reliability.
ASIC routers are capable of handling 100-1000x the throughput of the most over-provisioned Linux server (and that may even be an understatement). ASIC routers can also route packets with latency between 750us (0.75ms) and 10us (0.01ms!), complemented by multi-second (>GB) packet buffers.
QoS is rarely used at scale, if anything only on the access layer, because transit has become so cheap that ISPs have more bandwidth than they know what to do with. These days, if a link is congested, it's not cost saving, but instead poor network planning. QoS also has very limited benefits at >100G scale.
With that said, I feel that this article is definitely missing the full picture.
uint8_t *buf = ...; struct example_payload *payload = (struct example_payload *) buf;
That's why when you access the variables you need to byte order swap. This is absolutely not portable, I agree. I also agree that it is error-prone. However, it is the reality of a lot of performance critical software.
This article is a good demonstration of the performance improvements via mmap zero-copy: https://medium.com/@kaixin667689/zero-copy-principle-and-imp...
Netflix also relies on zero-copy via kTLS & zero-copy TLS to serve 400Gbps: https://papers.freebsd.org/2021/eurobsdcon/gallatin-netflix-...
However, the performance gap can get even larger! (The kernel is historically not great at this.) For NVME & packet processors, you can see an increase of 10,000%+ in performance easily via a zero-copy implementation. See: https://www.dpdk.org https://spdk.io
- mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order.
- filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.)
This is absolutely not "premature optimization." If you're a C/C++ engineer, you should know off the top of your head how many cycles syscalls & memcpys cost. (Spoiler: They're slow.) You should evaluate your performance requirements and decide if you need to eliminate that overhead. For certain applications, if you do not meet the performance requirements, you cannot ship.
The point is that our industry has a lot of opinionated individuals that tend to disagree on fundamentals, implementations, designs, etc., for good reasons! That's why we have thousands of frameworks, hundreds of databases, hundreds of programming languages, etc. Not everything our industry does is profit driven, or even rational.
At that point, it doesn't matter how many vantage points you verify from: all traffic goes to your hijack. It only takes a few seconds for you to verify a certificate, and then you can drop your BGP hijack and pretend nothing happened.
Thankfully there are initiatives to detect and alert BGP hijacks, but again, if your organization does not have a strong security competency, you have no knowledge to prevent nor even know about these attacks.
Bolting on extensions to existing protocols not designed to be secure, while improving the situation, has been so far unable to address all of the security concerns leaving major gaps. It's just a fact.