Securing your ssh server | Hacker News Reader

Securing your ssh server

rackerhacker.com

91 pointsnotmyname46 comments

I like to SSH in from all over the place, and so a default deny iptables policy would be a disaster for me. After some research, I went ahead and blocked all of China and Korea using iptables.

For anyone curious I put my modified iptables rules up at http://pastebin.com/UPL3w6FQ

Backstory:

I wrote a quick django app that watches fail2ban logs and plots a heatmap of where in the world the attacks are coming from. After watching this for a bit I found that most of these attacks originate from Chinese and Korean IP space. So I set out to block all of China and Korea.

After some googling, I found that okean.com maintains a handy SMTP Chinese/Korean iptables ruleset at http://www.okean.com/antispam/iptables/rc.firewall.sinokorea. I took their rules and modified it to drop all traffic from these IPs instead of just port 25.

Consider using ipset. Your ruleset is very huge and slows down your netfilter...

Thank you so much for the tip, I'll look at doing this.

FWIW though I have almost no traffic to these boxes, so I've never noticed any sort of performance issues.

Care to share the Django app? I'd like to explore doing this as well.

Those Asian addresses are only banging away at your server with weak u/p combinations, so if you've implemented PKA then they're already nothing to worry about. Blocking those IP addresses does little more than filters them out of one log (sshd) and into another (netfilter, assuming you're logging). The same goes for moving it to a non-standard port. You're only protecting yourself from the unsophisticated attackers looking for low-hanging fruit, which you've presumably mitigated with PKA already.

The author of the article advocates filtering to trusted sources because it's so much more effective. Ask yourself just how often you access your sshd from a new-to-you location. I'd be willing to bet that it isn't so often that you can't anticipate it. This effectively becomes a third factor in your authentication schema. It requires any attacker to have your private key, know it's passkey, and be at a pre-approved location. Gaining access to your sshd then necessitates attack vectors for your sshd, your private key or the encryption algorithm you used to generate it, and netfilter or whatever runs your firewall.

This ignores vectors like say, ubuntu/debian having an insecure keygen due to their own crappy custom patch to ssh, which was actually the case for quite a while.

I was glad that I was using openbsd with pf as my firewall when that came down.

I don't imagine there are too many Ubuntu installations in environments where this kind of SSH security is desirable. I could be wrong though, and you'd be right that this doesn't excuse them from shipping broken software. Yet broken software doesn't legitimize these frivolous measures. It only delegitimizes distributions like Ubuntu for settings like these.

Couldn't they use a vpn and tunnel in from any origin?

Incidentally, access to my server from China has proven indispensable. I was trying to read a bunch of blogs that were blocked by the great firewall while I was waiting for my plane at PVG, so I had to ssh in to my server and read them from there.

"Use a non-standard port: I'm not a big fan of security through obscurity and it doesn't work well for ssh."

When I had sshd running on the standard port on my MacMini, it would often slow down and start spinning the fan like crazy while some cracker was bruteforcing accounts and passwords. Now that I have it on a non-standard port, that never seems to happen anymore.

fail2ban (also mentioned in the article) would help you too..

That's good for your mac mini, but not a normal use case for a server running sshd (needing to be quiet, that is). Something that might also work for you would be to limit inbound access from known-trusted ip ranges, as well.

Running sshd on a nonstandard port isn't in itself a bad idea (other than the potential difficulty of logging into the server later if you forget the port or something), but it shouldn't be considered a strong security measure at all.

A normal use case for a server would not be keeping quiet, but conserving energy is very much on server people's mind.

Absolutely! However, I would argue that if you are looking to conserve energy, moving the sshd port is not something that should be at the top of your list. :)

When I ran OpenBSD I used to block on repeated failed attempts(see: http://home.nuug.no/~peter/pf/en/long-firewall.html#BRUTEFOR...).

I used to change ssh's port, but not anymore. If a bruteforce attack is a threat to you, then you are better being compromised by a random script kiddie or spammer than by a targeted attacker looking after your data. Take that scans as free, low skilled, penetration tests.

A nonstandard port removes a crazy amount of login attempts though for something as simple as adding:

    Host example.com
        Port #####

to your ~/.ssh/config

Alternately, adding those lines to every single host you control is one hell of an annoyance for something that doesn't stop any sort of real attack on your server. It's trivial for a real attacker to find your sshd running on any port.

Well sure it could be bad if you are using 10 computers or you have 10 different servers to login to, but personally I have 2 computers that I use and only manage a few servers so it isn't all that bad, plus the config file is linked to dropbox for me.

Besides it is more like a one time annoyance unless you are one of those crazy people that formats without backups every month.

Yes, this is true - it's not going to stop someone who really wants in from finding your SSH port. But at the very least, it's useful when looking through your logs, because the ONLY failed log in attempts you will see are from people who mean business.

Here is an app idea I had: a daemon runs, and it opens 20-30 ports. These ports simulate ssh/telnet/imap servers, at least for limited interactions, but do not actually do anything. They may also run slowly, to consume time on the part of the attacker.

When a user connects to these phony ports, their IP is blocked, or at least blocked from the ports which run legitimate services.

Are there any similar programs, or perhaps some flaw with this idea that I'm not seeing?

Yes; this is the basic idea behind honeyd, which is famous.

Thanks.

About slowing somebody down: http://en.wikipedia.org/wiki/Tarpit_%28networking%29

About automatic blocking: Be careful to not block legit users or services after a flood of spoofed packets to your $device.

Anecdote: I know I'm not a very big target, but disabling password authentication and using fail2ban has kept all of my servers and home machines safe and hacker free for multiple years now. Even during the Debian SSH key fiasco, fail2ban would lock out would-be brute-forcers early enough that they weren't able to exploit my weak SSH key before I could get home from vacation, regenerate my key, and distribute it to all my machines.

You can use simple iptables rules to block brute forcing, while still leaving things open for you on standard ports:

http://www.digitalsanctuary.com/tech-blog/debian/using-iptab...

Although now I've moved to CSF which does this and much much more.

I firewall off the SSH port for one minute after 4 unsuccessful logins in one minute:

    -A SSH_CHECK -m recent --set --name SSH --rsource 
    -A SSH_CHECK -m recent --update --seconds 60 --hitcount 4 --name SSH --rsource -j REJECT --reject-with icmp-port-unreachable 
    -A SSH_CHECK -p tcp -m tcp --dport 22 -j ACCEPT

This is in addition to default REJECT for everything that I don't explicitly allow.

(Incidentally, the kernel I'm running doesn't support this for ipv6 addresses... so you can bruteforce my ssh server over ipv6 for as long as you want. But fortunately, it's hard to automatically scan the entire ipv6 address space :)

I run a script every 15 minutes from cron that updates my iptables rules based on a whitelist of dynamic dns entries. That lets me logon from my laptop wherever I happen to be.

Two links I remember liking the last time I wrote iptables rules:

http://blog.andrew.net.au/2005/02/16#ipt_recent_and_ssh_atta...

http://www.la-samhna.de/library/brutessh.html

I use firehol (firehol.sf.net) along with knockd. Combined, it seems to be a rock-solid defense. The firewall will simply drop packets until someone enters the proper knock.

FireHOL looks like a neat tool, but seeing their last (official) release was in 2008 turned me off. I'm using Shorewall now, and as a plus, it supports ipset (mentioned by rw-).

Just a quick one on ports, changing them is a fantastic way to stop brute force attacks, and adding something like denyhosts reduces this even further. I used to have a 1024+ port for my ssh until I found out that if a regular user has access to your box already, if they can cause the SSH daemon to die somehow they can spawn a new process on that port (as 1024+ ports can be used by non-root users) and capture your password.

I know it's pretty tin foil hat of me but another layer is always good.

Use OSSEC ( http://ossec.net ) to monitor the logs/system for attacks. Not only brute force, but all sort of stuff.

*open source.

How should I do secure the ssh server on a Mac (running Snow Leopard)?

I tried using denyhosts (also mentioned in the article), but OSX 10.6 is ignoring /etc/hosts.deny, and it doesn't come with iptables. I do not want to disable passwords (i.e. use key-only authentication), since I tend to ssh to my home machine from various computers. And I can't change the port number due to firewall issues at my office.

Any other ideas?

OSX comes with ipfw (at /sbin/ipfw) which is an older tool much like iptables (on Linux) or pf (which I believe is more common than ipfw on BSD systems now). Since these tools are kernel-based and OSX's kernel is BSD-based rather than the Linux kernel, it makes sense that you find a BSD firewall tool rather than iptables.

http://en.wikipedia.org/wiki/Ipfirewall

http://en.roolz.org/Blog/Entries/2009/11/6_OpenBSD_PF_on_Mac...

I just came across this, but you might also want to take a look at Sshguard.

http://www.sshguard.net/

If denying SSH isn't an option, you could try PortKnocking. It's not secure in itself, but it's useful if you're wanting to dynamically allow IPTables access to SSH. I knocked (no pun intended) up a quick Perl script to implement this on Redhat based systems: http://jdlawrie.co.uk/scripts/PortKnocker.txt

I just started using gpg-agent to do my ssh authentication via my crypto-stick (An OpenPGP smartcard with the reader built in.) I'm totally sold on it. It's got a hardware lockout so it can't be brute forced. The actual private key never touches the client machine's memory. And its reasonably cheap.

More info on the smartcard pls.

http://www.privacyfoundation.de/crypto_stick/crypto_stick_en...

Basically they offload your private keys to a dedicated device, completely in hardware, that's isolated from the machine's memory, so they can't be compromised via a software attack. The only attacks I've ever seen used to retrieve the primary key involve cracking the case and trying to open the chips up physically.

The cards are OpenPGP, and have room for three keys, up to 3072 bit RSA. The first two are pretty standard PGP keys, a signing key and and encryption key.

The third doesn't get used as much. It's a 'certification' key. If you have a certification key, you can use gpg-agent as a drop-in replacement for ssh-agent. And then you don't need an id_rsa file or anything like that stored on your computer. It's all on the card.

When you ssh into a box that has your public key setup, gpg-agent asks for your passphrase. Once that's done, the session key and that stuff gets setup on the smartcard. Since all that happens on the card, it doesn't matter if your system is compromised, the key can't be retrieved. (It also invalidates that old XKCD joke about getting your password with a rubber hose. If you destroy the device, your password is useless.)

The cards also lock out after three invalid passwords, so they can't be brute forced. If you do lock it out, there's an admin password to remove the lock. If someone gets that wrong three times, it actually erases the keys and turns the device into a brick.

It's really way more secure than I need, but it's a neat little gadget.

Anyway, I've seen dozens of articles over the years talking about securing a server's ssh config. But I've never seen one that addressed securing the clients...

Thanks. I'll look into it. Never used gpg-agent before so will have to research.

Sounds like a good blog post on how you set things up if you do maintain a blog somewhere. =D

What's the best way of dealing with the chicken-and-egg problem of initial login when using PKA? Should login be denied until the private key is uploaded via something other than ssh?

No, you should do it via SSH, just try to do it as soon after standing up your server as possible.

The only place I know of that gets this right is Amazon, who stands it up with a key (admittedly one of theirs) that they force you to download before you can connect the first time.

You can now import your own ssh key. http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineR...

that's how I do it. I make you email me a public key before I set you up, then I use that public key to control access to your serial console.

Right now my big problem is that the channel for the public key is email, which is no good.

On larger deployments I've always put the ssh pubkeys for authorized users in a package that is installed when the server is installed.

For what it's worth, changing port 22 to something "obscure" is an excellent idea: it separates the script kiddies and bots from real hackers trying to pwn the box. Your log files will contain only hack attempts that should be of concern.

Comments and critiques regarding the following script are most welcome.

  #!/bin/bash
  
  # Rules:
  # http://www.newartisans.com/2007/09/neat-tricks-with-iptables.html
  # Sel also:
  # https://help.ubuntu.com/community/IptablesHowTo
  
  echo "[SCRIPT] Limit to ports 1222 and 80."
  iptables -F
  iptables -P INPUT ACCEPT
  iptables -P OUTPUT ACCEPT
  iptables -P FORWARD ACCEPT
  iptables -A INPUT -i lo -j ACCEPT
  iptables -A INPUT -p icmp --icmp-type any -j ACCEPT
  iptables -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
  iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 1222 -j ACCEPT
  iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport www -j ACCEPT

  iptables -A INPUT -j REJECT --reject-with icmp-host-prohibited
  iptables -A FORWARD -j REJECT --reject-with icmp-host-prohibited
  
  echo "[SCRIPT] Ban incorrect SSH login attempts (120 seconds)."
  iptables -A INPUT -i lo -p all -j ACCEPT
  iptables -A OUTPUT -o lo -p all -j ACCEPT                    
  iptables -A INPUT -i eth1 -m state --state ESTABLISHED,RELATED -j ACCEPT
  iptables -A INPUT -p tcp ! --tcp-option 2 -j REJECT --reject-with tcp-reset
  iptables -I INPUT -p tcp --dport 1222 -i eth0 -m state --state NEW \
    -m recent --set
  iptables -I INPUT -p tcp --dport 1222 -i eth0 -m state --state NEW \
    -m recent --update --seconds 120 --hitcount 4 -j DROP
  iptables -P INPUT DROP
  
  echo "[SCRIPT] Drop spoofed IP addresses."
  iptables -A INPUT -s 10.0.0.0/8     -j DROP
  iptables -A INPUT -s 169.254.0.0/16 -j DROP
  iptables -A INPUT -s 172.16.0.0/12  -j DROP
  
  iptables -A INPUT -s 224.0.0.0/4      -j DROP
  iptables -A INPUT -d 224.0.0.0/4      -j DROP
  iptables -A INPUT -s 240.0.0.0/5      -j DROP
  iptables -A INPUT -d 240.0.0.0/5      -j DROP
  iptables -A INPUT -s 0.0.0.0/8        -j DROP
  iptables -A INPUT -d 0.0.0.0/8        -j DROP
  iptables -A INPUT -d 239.255.255.0/24 -j DROP
  iptables -A INPUT -d 255.255.255.255  -j DROP
  
  echo "[SCRIPT] Limit spamming PINGs."
  iptables -A INPUT -p icmp -m icmp --icmp-type address-mask-request -j DROP
  iptables -A INPUT -p icmp -m icmp --icmp-type timestamp-request -j DROP
  iptables -A INPUT -p icmp -m icmp -m limit --limit 2/second -j ACCEPT
  
  echo "[SCRIPT] Drop packets with an invalid state."
  iptables -A INPUT   -m state --state INVALID -j DROP
  iptables -A FORWARD -m state --state INVALID -j DROP
  iptables -A OUTPUT  -m state --state INVALID -j DROP
  
  echo "[SCRIPT] Drop SYN,FIN invalid ordering."
  iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,FIN SYN,FIN -j DROP
  iptables -A INPUT -p tcp -m tcp --tcp-flags SYN,RST SYN,RST -j DROP
  
  echo "[SCRIPT] Limit RST RST spam."
  iptables -A INPUT -p tcp -m tcp --tcp-flags RST RST \
      -m limit --limit 2/second --limit-burst 2 -j ACCEPT
  
  echo "[SCRIPT] Restrict port scanners for 24 hours."
  iptables -A INPUT   -m recent --name portscan --rcheck --seconds 86400 -j DROP
  iptables -A FORWARD -m recent --name portscan --rcheck --seconds 86400 -j DROP
  
  iptables -A INPUT   -m recent --name portscan --remove
  iptables -A FORWARD -m recent --name portscan --remove
  
  echo "[SCRIPT] Drop scans on port 139 (Microsoft)."
  iptables -A INPUT   -p tcp -m tcp --dport 139 \
      -m recent --name portscan --set -j LOG --log-prefix "Portscan:"
  iptables -A INPUT   -p tcp -m tcp --dport 139 \
      -m recent --name portscan --set -j DROP
  
  iptables -A FORWARD -p tcp -m tcp --dport 139 \
      -m recent --name portscan --set -j LOG --log-prefix "Portscan:"
  iptables -A FORWARD -p tcp -m tcp --dport 139 \
      -m recent --name portscan --set -j DROP
  
  iptables-save > /etc/network/iptables
  printf '#!/bin/sh\niptables-restore < /etc/network/iptables\n' > /etc/network/if-pre-up.d/iptables
  chmod 754 /etc/network/if-pre-up.d/iptables

Reply on news.ycombinator.com