HNHacker News
TopNewBestAskShowJobs

iscoelho

684 karma · joined January 18, 2017

submissionscomments
iscoelho··on Why do we need DNSSEC?
"Most domains". Yes, it is possible that nobody bothers to DNS hijack your domains. Sadly I've worked for organizations where it did happen, and now they have DNSSEC.
iscoelho··on Why do we need DNSSEC?
You are again ignoring the fact that DNSSEC would have prevented a $152,000 hack. Yes, we are aware organizations are not always serious about security. For those that are though, DNSSEC is a helpful tool.
iscoelho··on Why do we need DNSSEC?
"I do like that the threat model you propose is coherent only for sites behind Cloudflare, though."

The threat model I proposed is coherent for Cloudflare because they have done a lot of engineering to make it almost impossible to globally BGP hijack their IPs. This makes the multi-perspective validation actually help. Yes, other ISPs are much more vulnerable than Cloudflare, is there a point?

You are not saying DNSSEC doesn't serve a real purpose. You are saying it is annoying to implement and not widely deployed as such. That alone makes me believe your argument is a bit dishonest and I will abstain from additional discussion.

iscoelho··on Why do we need DNSSEC?
Yeah I'm by no means saying the implementation is good. RPKI is a joke as well in my opinion. But it's all we have right now.

I am saying it is dishonest to discount the real security threat of not having DNSSEC.

iscoelho··on Why do we need DNSSEC?
TLS doesn't provide any security in this case because TLS certificates are generated based on DNS. See Lets Encrypt.
iscoelho··on Why do we need DNSSEC?
There are 2 ways to pull off this attack:

1. Hijack the HTTP/HTTPS server. For some IP ranges, this is completely infeasible. For example, hijacking a CloudFlare HTTP/HTTPS range would be almost impossible theoretically based on technical details that I won't go through listing.

2. Hijack the DNS server. Because there's a complete apathy towards DNS server security (as you are showing) this attack is very frequently overlooked. Which is exactly why in the cited incident attackers were capable of hijacking Amazon Route53 with ease. *DNSSEC solves this.*

If either 1 or 2 work, you have yourself a successful hijack of the site. Both need to be secure for you to prevent this.

iscoelho··on Why do we need DNSSEC?
As you should be aware, multi-perspective validation does not solve anything if your BGP hijack is accepted to be global. You will receive 100% of the traffic.

DNSSEC does greatly assist with this issue: It would have prevented the cited incident.

iscoelho··on Why do we need DNSSEC?
I would say it is more a testament to the unfortunate state of cybersecurity. These "theoretical" attacks happen. Everyone just thinks it won't be them.
iscoelho··on Why do we need DNSSEC?
It is important. This is unfortunate rhetoric that is harming the safety of the internet.

"For instance, in April 2018, a Russian provider announced a number of IP prefixes (groups of IP addresses) that actually belong to Route53 Amazon DNS servers."

By BGP hijacking Route53, attackers were not only able to redirect a website to different IPs, globally, but also generate SSL certificates for that website. They used this to steal $152,000 in cryptocurrency. (I know I know, "crypto", but this can happen to any site: banking, medical, infrastructure)

Also, before you say, RPKI doesn't solve this either, although a step in the right direction. DNSSEC is a step in the right direction as well.

[1] https://www.cloudflare.com/learning/security/glossary/bgp-hi...

iscoelho··on Using AZs can eat up your budget – From Prometheus to VictoriaMetrics
I agree with this post that Hetzner is a bad example. They are focused on a budget deployment.

I do not agree that a state-of-the-art high capacity deployment is as expensive as you think it is. If an organization pays MSRP on everything, has awful procurement with nonexistent negotiation, and multiple project failures, sure, maybe. In the real world though, we're not all working for the federal government (-:

iscoelho··on Using AZs can eat up your budget – From Prometheus to VictoriaMetrics
In Ashburn, VA, I can buy Dark Fiber for $750 MRC to any datacenter in the same city. I can buy Dark Fiber for $3-5K MRC to any random building in the same city.

That Duplex Dark Fiber with DWDM can run 4TBPS of capacity at 100GE (40x 100GE). Each 100GE transceiver costs $2-4K NRC dependent on manufacturer - $160K NRC for 40x. (There are higher densities as well, like 200/400/800GE, 100GE is just getting cheap.)

In AWS, utilizing 1x100GE will cost you >$1MM MRC. For significantly less than that, let's say absolutely worst-case 5K MRC + 200K NRC, you can get 40x100GE.

Now you have extra money for 4x redundancy, fancy routers, over-spec'd servers, world-class talent, and maybe a yacht if your heart desires.

iscoelho··on How much does The Cloud really cost?
It's a great start, but it's missing the silent killer: data transfer!

AWS egress data transfer pricing is 100-1000x the price of equivalent IP Transit. (Yes, I am not exaggerating.) It is so significant that I've seen data transfer account for >90% of 7-figure AWS bills. Given enough spend, that alone is enough to make the business case for a migration.

iscoelho··on How much does The Cloud really cost?
Exactly! When your savings are 2-3x per unit, you have more room in the budget to over-provision. Now you have increased headroom for growth, and improved redundancy.
iscoelho··on Law enforcement takes down 'DDoS-for-Hire' sites in Operation PowerOFF
The largest DDoS attacks recorded are un-amplified and un-spoofed. The sources are malware (see Mirai and its variants) via legitimate internet connections.

Even the largest ISPs generally do not have enough spare capacity to just "block UDP packets," as the largest attacks are now measured in Tbps. There is also nothing stopping attackers from just sending TCP packets.

As consumer internet speeds grow, DDoS grows.

iscoelho··on You Don’t Know Jack about Bandwidth
That is almost impossible pricing these days at any bandwidth level unless you are either a) renewing an existing contract without negotiation b) are negligently bad at negotiation and are being taken advantage of c) are self dealing / sabotaging. (*unless you're in Australia)

Bandwidth is cheap.

iscoelho··on You Don’t Know Jack about Bandwidth
This article appears to be written from a solid Linux networking background, but not from an ISP networking background.

ISPs at scale do not use software routers. They use ASIC routers (Juniper/Arista/Cisco/etc.), for many reasons 1) features 2) capacity 3) reliability.

ASIC routers are capable of handling 100-1000x the throughput of the most over-provisioned Linux server (and that may even be an understatement). ASIC routers can also route packets with latency between 750us (0.75ms) and 10us (0.01ms!), complemented by multi-second (>GB) packet buffers.

QoS is rarely used at scale, if anything only on the access layer, because transit has become so cheap that ISPs have more bandwidth than they know what to do with. These days, if a link is congested, it's not cost saving, but instead poor network planning. QoS also has very limited benefits at >100G scale.

With that said, I feel that this article is definitely missing the full picture.

iscoelho··on The Byte Order Fallacy
100% on the business implications. Although a lot of engineers never have to touch it, DMA (& zero-copy) implementations are foundational to the performance of modern day computers that we sometimes take for granted.
iscoelho··on The Byte Order Fallacy
The performance would be absolutely horrendous if network drivers were programmed this way. DMA (Direct Memory Access) is all about avoiding deserialization and copies of the data.
iscoelho··on The Byte Order Fallacy
Haha, it's zero-copy! I never said it was "faster-copy" (-:
iscoelho··on The Byte Order Fallacy
The purpose of zero-copy can be to avoid deserialization at all. All you do to deserialize is:

uint8_t *buf = ...; struct example_payload *payload = (struct example_payload *) buf;

That's why when you access the variables you need to byte order swap. This is absolutely not portable, I agree. I also agree that it is error-prone. However, it is the reality of a lot of performance critical software.

iscoelho··on The Byte Order Fallacy
memcpy is extremely slow. On any high-load Linux webserver, you can type "perf top" and see 20%~ of the CPU usage consumed by memcpy/syscalls/virtual memory.

This article is a good demonstration of the performance improvements via mmap zero-copy: https://medium.com/@kaixin667689/zero-copy-principle-and-imp...

Netflix also relies on zero-copy via kTLS & zero-copy TLS to serve 400Gbps: https://papers.freebsd.org/2021/eurobsdcon/gallatin-netflix-...

However, the performance gap can get even larger! (The kernel is historically not great at this.) For NVME & packet processors, you can see an increase of 10,000%+ in performance easily via a zero-copy implementation. See: https://www.dpdk.org https://spdk.io

iscoelho··on The Byte Order Fallacy
If you are using C/C++ for any new app, there is a possibility you are writing code that has a performance requirement.

- mmap/io_uring/drivers and additional "zero-copy" code implementations require consideration about byte order.

- filesystems, databases, network applications can be high throughput and will certainly benefit from being zero-copy (with benefits anywhere from +1% to +2000% in performance.)

This is absolutely not "premature optimization." If you're a C/C++ engineer, you should know off the top of your head how many cycles syscalls & memcpys cost. (Spoiler: They're slow.) You should evaluate your performance requirements and decide if you need to eliminate that overhead. For certain applications, if you do not meet the performance requirements, you cannot ship.

iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
See my post above about BGP hijacks: https://news.ycombinator.com/item?id=41511582 - They're way easier than you think.
iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
Do you use tabs or spaces? Just joking, but:

The point is that our industry has a lot of opinionated individuals that tend to disagree on fundamentals, implementations, designs, etc., for good reasons! That's why we have thousands of frameworks, hundreds of databases, hundreds of programming languages, etc. Not everything our industry does is profit driven, or even rational.

iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
100% - another for the BGP hijack!
iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
BGP hijacks unfortunately completely destroy that. RPKI is still extremely immature (despite what companies say) and it is still trivial to BGP hijack if you know what you're doing. If you are able to announce a more specific prefix (highly likely unless the target has a strong security competency and their own network), you will receive 100% of the traffic.

At that point, it doesn't matter how many vantage points you verify from: all traffic goes to your hijack. It only takes a few seconds for you to verify a certificate, and then you can drop your BGP hijack and pretend nothing happened.

Thankfully there are initiatives to detect and alert BGP hijacks, but again, if your organization does not have a strong security competency, you have no knowledge to prevent nor even know about these attacks.

iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
The point is that, ironically, new standards may have been a better option.

Bolting on extensions to existing protocols not designed to be secure, while improving the situation, has been so far unable to address all of the security concerns leaving major gaps. It's just a fact.

iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
The comic is about re-inventing the wheel. What you propose "standards evolving" would be the opposite in spirit (and is what has happened with DNSSEC, RPKI, etc)
iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
As long as TLS/SSL depends on Internet PKI as it is, it is flawed. I guess there's always Private PKI, but that's if you're not interested in the internet (^:
iscoelho··on We spent $20 to achieve RCE and accidentally became the admins of .mobi
Let's not fool ourselves by saying we're purely profit driven. Our industry argues about code style (:
← PreviousPage 4 of 8Next →