It might be a firewall issue or one of the Fail2Ban jails. If you're using all the Nginx jails, try disabling them and see if it fixes the issue.