Comment by rstuart4133 | Hacker News Reader

Comment by rstuart4133 | Hacker News Reader

Parent Full thread

You're being far too polite. He's just plain wrong on this point:

> do not change your standard ports. do not use fail2ban.

That used to be my attitude too. But reality intruded.

Example: I ran a SIP Server for a business. SIP Servers are very attractive targets because if you get past the security, you can make then call paid numbers and make real money. Knowing that, I used huge security tokens. No one got past them. But - it didn't stop them from trying. 100's of times per second at times. The CPU load created by processing the login attempts was so great the SIP server started dropping calls. As it happens, asking the kernel to drop packets from miscreant IP's takes far, far less CPU. Fail2ban to the rescue.

Example: Move on a few years, and now spammers costs have dropped so much it is worth their while to target IMAP, POP and SMTP servers. The story is the same: they never got through, but despite that their attempts increased to the point the server became overloaded. I can't listen on a different port. Fail2ban to the rescue.

Example: Move on to the last couple of years. I move my personal stuff to a VPS. To save costs it's very memory constrained. A problem soon arises. After a few weeks of trouble free operation it keeps slowing down, and sometimes becoming unresponsive. Turns out it's running out of memory; the OOM killer is running amok. That's odd, as there should be almost no load on this thing. Well, apart from ssh login attempts it seems. Turns out there is one of them every few seconds. They have no hope of succeeding of course as I only allow pubic key auth, but the load it creates is a problem. Fail2ban to the rescue.

I don't see anything wrong with the principle, which is you block these attacks at the cheapest point possible. That happens to be the point of entry, before you spend CPU cycles processing the packet (even in TCP). Fail2ban leverages that by using the kernel's networking stack. It's forced to glean the information on who to ban from log files, which is messy, error prone and fragile. I don't like that either, but I'll take what I can get.

> I can't listen on a different port.

And the author insists you shouldn't change ports any way. But seems to have no trouble just saying "use the security in your applications". Well... what sort of security is there in an IMAP server beyond "validate credentials"?

My other thought, reading the author's post, and having seen enough of this type of attitude, is that this type of person would be the one to say "just use... <external mail service>" or similar. "You can't ever do better than them!" Well... I have to suspect that techniques like fail2ban - blocking network access after X suspicious access attempts - get employed at the big service providers too. Possibly not fail2ban specifically, but the same ideas.

Several years ago I was part of team managing one of the busiest recursive DNS resolvers in the country. Around the time, DNS reflection attacks were becoming all the rage (I think there might have been a vulnerability in PowerDNS to blame, I don't specifically recall the details).

The solution we employed was a "dumb" fail2ban like script to ban any abusive IP address. It worked wonders and practically completely neutralized the attacks. We received zero complaints from any legitimate users of the service.

If I recall correctly, even after the attacks subsided (CVEs was patched etc.), we still kept a toned down version of the script in place.

> at best, fail2ban: does nothing

This is just plain wrong. Even if you're only using key based auth and there's no way an attacker could have your keys, fail2ban will still block the abusive traffic.

I have no issues with blocking abusive traffic instead of letting the attackers beat their fists against my armored doors.

> at worst, fail2ban: causes you massive inconvinience or total lockout

The authors main criticism seem to be that you might lock yourself out of your own server one day. This hardly seems fair because the author is saying "just setup SSH correctly and you won't have any issues" while also saying "if you don't setup fail2ban correctly, you'll have a whole bunch of issues!" OK, so shouldn't we just setup fail2ban correctly as well?

Having said all that, my recommendation for people who are going to use an indiscriminate fail2ban setup on SSH access would be to have some out-of-band access in case you do happen to ban yourself, even if it's less convenient to use than SSH.

> The authors main criticism seem to be that you might lock yourself out of your own server one day.

I wonder if the author has ever used KVM, iDrac, IPMI, etc., or the log-in consoles on cloud platforms.

And if you don't have those then 1) woe be to you, and 2) pay the 150/hr remote hands fees to get someone to console in for you.

Yeah, they also miss (IMO) one of the biggest advantages of using something like fail2ban which is the reduction in logging noise. I've seen multiple VMs effectively DoS-ed because their disk filled up with SSH log entries. I've been paged by security monitoring tools that registered all the failed login attempts as attacks, and I've had to filter out or just wade through all those logs when investigating other issues.

fail2ban also works with services other than SSH, and one of my favorite tricks is to point it at nginx logs, add a `robots.txt` with a honeypot entry and have fail2ban block any IP that hits that or any of a dozen common vulnerability endpoints that I know aren't on the server (like `wp-admin.php`). Keeps the web logs cleaner and more useful.

I too only allow pubic key auth :P

More seriously, I don’t see why anyone would want to disable fail2ban (outside of purposefully being against the grain) because it seems like “free” protection. I’m sure there’s some situations where it may be beneficial to, but those seem like extreme edge cases.

Stories like this make me wonder if we're truly better off with anonymous Internet traffic.

(I'm not advocating for a change; I'm just unfamiliar with the arguments for both sides.)

fwiw, I'm not trying to be anonymous. I've updated my profile to make that plain.

The examples where there to demonstrate the conclusion in the final paragraph matters in real life. The para could stand on it's own - it's just a statement of the obvious. But like all obvious things, it's only obvious once you see it. Like the author of the article, I didn't see it for years (decades?).

> fwiw, I'm not trying to be anonymous. I've updated my profile to make that plain.

Were you thinking that I was arguing that e.g. you shouldn't be able to use a pseudonym to post comments?

Sorry if that's how my comment reads. The thing I was wondering about was if/why we should allow spammers / crackers to retain their anonymity when attacking computers in the way you described. E.g., 100 attacks / second on some port, with no accountability for the persons who originated / propagated those packets.

I'm curious about the pros/cons of law-enforcement agencies having the ability (and mandate) to hunt down the sources of such traffic.

I realize that doing so would open a huge can of works, so I'm 10000% not advocating for it. I'd just like to understand the steel-man arguments for and against.

It's no use if you know your attackers are from SE Asia or Russia. It just creates another incentive for corporations to hoard use data and sell it. Block them and go on with your life. They have an IP address, so it's not 'anonymous'. Law enforcnent could probably get to them if it's worth it and they try hard enough.

Reply on news.ycombinator.com