Design behaviour here was on bad input, shutdown. Some bad input happened, thus shutdown happened. Seems fine to me. I'm not seeing a grave flaw.
Effectively the impacted instances are now no longer able to do their job, they cannot propagate more rules.
That's the flaw.
When they receive something that they don't understand they need to propagate without breaking connection, and they do not.
And this is how things are done, per the article:
> On 2 June 2023, a small Brazilian network (re)announced one of their internet routes with a small bit of information called an attribute that was corrupted. The information on this route was for a feature that had not finished standardisation, but was set up in such a way that if an intermediate router did not understand it, then the intermediate router would pass it on unchanged.
> As many routers did not understand this attribute, this was no problem for them. They just took the information and propagated it along.
However, the information was corrupted at the source, and those devices that did understand the attributed detected the corruption and rejected it:
> However it turned out that Juniper routers running even slightly modern software did understand this attribute, and since the attribute was corrupted the software in its default configuration would respond by raising an error that would shut down the whole BGP session.
The question is whether the session shutdown was appropriate (at least by default).
I think Juniper's default is reasonable, as attributes can communicate important information about the intentions of the network operators, and rejecting a particular attribute and basically ignoring the intentions of the operators may also cause problems.
I also think that thinking that perhaps allowing the session to continue but throwing a (loud) warning is also reasonable (and there is a flag for that).
It is nothing like that.
HTTP requests do not propagate across an organization's entire network, or possibly the entire Internet.
> Or, for an even better analogy, it is like an HTTP server that, upon receiving a malformed HTTP request, drops all current and future traffic from that IP.
So just like (e.g.) fail2ban blocking IPs that continuously send bad login requests? That's awesome! I would want my web server to block bad clients.
Heck, it would be nice if web browsers would do the same thing with content they downloaded. In the early days of the web there were all sorts of garbage files out there and instead of forcing people to correct their HTML they tried to be clever in mind reading:
* https://en.wikipedia.org/wiki/Tag_soup
For a time there was an entire 'movement' to get people to clean up their act:
* https://en.wikipedia.org/wiki/W3C_Markup_Validation_Service
* https://en.wikipedia.org/wiki/HTML_Tidy
It's fine to have a flag to have stringency be a policy that can be toggled, but I think Postel's law ("be liberal in what you accept") can cause issues over the long-term:
Well, somewhat, except that it's not awesome. It's like fail2ban, but implemented on a server that sits behind a load balancer. So, when it receives a bad login request from an IP and it bans that IP, it actually banned the IP of the load balancer, so now it won't receive any other requests. So that a malicious client can just send 1 bad request, and take down everyone else using the whole server.
Basically, this whole discussion is not so much about Postel's law, it's about limiting failure domains. A bad route advertisement should make that 1 route inaccessible. It is not good design for a bad route advertisement to take down all the other good routes. It is particularly bad design for a protocol where routes are automatically propagated by devices that don't think there's anything wrong with them.
If you want to compare it to the HTML situation, the bug can also be seen as a browser that, when it receives invalid HTML from www.google.com/search would not only throw an error, but also refuse any other connection to *.google.com/*. The proposed fix is not to interpret anything as valid HTML. It is to show an error only when accessing www.google.com/search.
No they did not, they shutdown a whole session down for a malformed announcement. The right decision would have been to ignore the corrupt attribute and maybe not install that particular route.
A logged message saying "Attribute 129 with value 0 is invalid" would have helped operators more than just blindly shutting down sessions through which this was announced.
I think Juniper's default is reasonable: rejecting a particular attribute and basically ignoring the intentions of the operators could also cause problems, and instead of trying to read the minds of those running the networks, the software doesn't try to be clever.
I also think that thinking that perhaps allowing the session to continue but throwing a (loud) warning is also reasonable (and there is a flag for that).
My point still stands: attributes can have important information about the intentions of how a network operator wishes their network to be seen to the rest of the world. Having a corrupt attribute, and thinking it's okay to simply assume that without that attribute the rest of the advertisement is still valid is not fine IMHO: you're basically trying to guess/assume the intentions of the source of the advertisement.
For all we know ignoring that attribute and accepting the rest of the advertisement as-is could have caused flood of traffic over the connection causing a just-as-effective-DoS.
But! All your other routes that you advertised earlier with perfectly fine attributes will continue working. This is what prevents this from becoming a DoS vulnerability.
Let's take a more complete example.
ISP1 advertises 2 routes to ISP2:
35.67.0.0/16 attr1=val1
37.61.0.0/16 attr1=corr
35.67.0.0/16 attr1=val1
37.61.0.0/16 attr1=corr
78.0.0.0/8
After the fix, ISP3 will keep routing traffic to 35.67.0.0/16 (respecting the value val1 for attribute attr1) and to 78.0.0.0/8 through ISP2. It will not route traffic to 37.61.0.0/16.
In the scenario that started off the author’s investigation, the bad attribute kicked COLT off the Internet even though COLT did not connect directly to the Brazilian ISP that sent the bad route update. When they received the bad update, COLT’s routers dropped their BGP sessions with intermediate ISPs that were unrelated to the Brazilian ISP, causing complete loss of connectivity.
If COLT’s routers had treated the error as a route withdrawal, they would have lost connectivity to the Brazilian ISP but not the rest of the Internet.
Maybe switch off on request is a dubious design choice for infrastructure that you'd rather stay alive, but it sounds like a new version of the spec has already been released to make that an option.
If you shutdown the whole session because of one bad forwarded message, you are indeed a broken implementation.
If it were just a Juniper implementation flaw it’d be less interesting than explaining an aspect of BGP error handling more generally which is indeed what the article was, for those of us aware of BGP but not familiar with specific protocol design decisions and implications.
If you quibbled with the word “Grave” I might agree since it’s not industrywide and impacting major vendors like Cisco and Huawei and those implementing RFC 7606 (although there is no patch for Nokia or Extreme or Bird!) I’d counterpropose that a more neutral TLDR title/summary for an HN audience might be “Grave flaws in non compliant BGP error handling.”
But on balance I think the author is better off with raising the alarm with a stronger headline; he clearly put his passion to good use and I learned a little something as a result.
Absolutely not. To me, it's a miracle that the internet even works.