Comment by crispyambulance | Hacker News Reader

Comment by crispyambulance | Hacker News Reader

crispyambulance

Just some really basic questions since I don't understand all the jargon in the article, can someone help me?

* They're talking about 100Gbps ethernet traffic. Where does one even see that line rate? It seems those rates would only occur at carriers, optical transport, or in very large data-centers.

* Intrusion detection, AFAIK, would mean that the packets have to be parsed and inspected. But most traffic is encrypted, right? How far beyond typical firewall capability can you get with this stuff if you can only look at the unencrypted part of the packet?

* Not sure exactly what "intrusion" means in this context? What are they looking for? What are some common rules that get applied? Every protocol is going to be wildly different and working at an application level. Don't you need specific application experts to even think about writing such specialized rules?

* There are commercial products which, I think, do this stuff from Gigamon and Netscout. These products have special FPGA-based switches that have many 10G, 40G or 100G ports, how does the tech in the article differ? What's the use case?

Sorry to be that harsh, but you can read everything in the net. For what a IDS is, start with that article it alone answers 3 of your questions:

https://en.wikipedia.org/wiki/Intrusion_detection_system

No offense taken. The wikipedia article makes some of the jargon clearer, but it still doesn't explain what is actually meant by intrusion detection, or even an example of a rule that's more than what a firewall would enforce. It talks about statistics and machine learning being used to apply rules. I am asking what kinds of rules? Keywords in the payload? Peculiar GET requests? Connections from Kazakhstan between midnight and 4am? Wouldn't rules like that have lots of false positives and piss off legit users?

In particular, I am curious about encrypted traffic (eg TLS/https/ssh). Wikipedia says IDS's can't work on encrypted traffic. "...Encrypted packets are not processed by most intrusion detection devices..."

But most traffic IS encrypted. So where do you ever have a situation where you have 100Gbps on a line, unencrypted? What's the use-case?

For the purposes of intrusion detection, even metadata about an encrypted connection can be helpful. You get the (potentially unencrypted) metadata about the connection at the beginning of it, but also a host of other metadata such as time of connections, amount of data transfer, volume/frequency of connections, etc. So even without knowing the contents, you can pick up on anomalous and suspicious activity on your network. You also get information such as the IP addresses of the endpoints and potentially hostname, which gets sent in cleartext during the SSL handshake so a server with virtual hosts and potentially multiple websites know which one is being requested and knows which certificate is appropriate to use.

For traffic inspection purposes, you can still monitor the contents of (most) encrypted packets, but have to have your own certificate on the end user's device. This is most common on corporate networks, where a company will either install their own certificate as part of their device management process or rely on third party apps like Bluecoat[1] to do that.

Once they've added their own certificate to your device's trust store, that can be used to decrypt and re-encrypt traffic in transit, without any overt indications to end users that it's being done (although you can sometimes detect if this is being done). There are ways to guard against this, such as certificate pinning[2], but that's up to each individual service setting up rather than something the end-user can do themselves.

In both cases, enterprise networks are generally one of the bigger users of such software. There are open source IDS's you can use on your home network, which won't require 100Gbps. But 100Gbps is easily achievable on an office network, particularly as you start to scale to multiple offices (and if you include intranet traffic, and not just processing outbound/external traffic).

[1] https://en.wikipedia.org/wiki/Blue_Coat_Systems

[2] https://carvesystems.com/news/cert_pin/

To learn about Intrusion Detection Systems (IDS) I found Unifi's documentation quite helpful. It's specific for their product but most of it is applicable elsewhere.

https://help.ui.com/hc/en-us/articles/360006893234-UniFi-USG...

> What are some common rules that get applied?

This is specifically answered under "Categories and Their Definitions":

> Compromised: This is a list of known compromised hosts, confirmed and updated daily as well.

> Scan: Things to detect reconnaissance and probing. Nessus, Nikto, portscanning, etc. Early warning stuff.

> SpamHaus: This ruleset takes a daily list of known spammers and spam networks as researched by Spamhaus.

> Web Apps: Rules for very specific web applications.

OK, thanks! so it can be like a firewall except dynamically updated by subscribing to some kind of service? This would work even for encrypted traffic.

Some systems use a CA cert on endpoints and a CA key on the web proxy portion of the IDS, so it's able to intercept and decrypt all traffic that passes through. This isn't as common anymore since it has massive security risks associated with a 'godmode' TLS cert.

But there are still ways to inspect encrypted traffic. TLSv1.2 and lower expose the SNI in the handshake. Even on TLSv1.3 there is plenty of metadata exchanged during the TLS handshake. For example, the JA3 spec [1] is a standardized hash of selected connection parameters such as cipher suites, extensions, curves, etc. It can reveal a decent amount of information about the software initializing the connection, and when compared to a list of known good or known harmful JA3's can flag connections as malicious.

[1] https://engineering.salesforce.com/tls-fingerprinting-with-j...

Core networks of cloud providers (FANG) are made up of many aggregated or multipath 100G/400G.

For one, there are many variations of geographical redundancy where there is no single machine that touches all of the traffic. Having everything behind a pair of redundant load balancers is somewhat antiquated.

> They're talking about 100Gbps ethernet traffic. Where does one even see that line rate?

The higher the peak throughout, the more margin you have for user applications.

If the peak throughout is 100Gbps then it’s likely that it will only consume 1% of resources at a more pedestrian 1Gbps throughout.

To answer your first question - I’ve personally achieved the advertised 100Gbps line rate on the bigger AWS instances.

Reply on news.ycombinator.com