Better yet, put a firewall in front of sshd and only allow connections from your management network ;)