Moving the port reduces security. Port 22 is a privileged port. Standard users can’t listen on port 22. If you move the port to 2222 or wherever, then if an attacker with local access can get sshd to crash, they can run their own sshd instead. If you left the port as the default, they wouldn’t be able to do that without chaining it with a privilege escalation attack. But because you changed the port, you disabled this security feature and it could become a privilege escalation attack.