Nothing would happen, ssh is designed to be open to the world. Using tailscale or a vpn to hide your IP is fine, but using tailscale ssh maybe not.
And one simple mistake, and we're screwed
There were like thousands of requests in an hour, and that went on continuously, before I changed the port.
Once I changed the ssh port to a large number, the OOM and heavy CPU usage stopped, and never came back. So, I think I'm safe, though I keep an eye on the logs, and for any unknown processes, but never seen anything out of ordinary.
The 64gb machine is my dev machine, as my IDE(intellij) runs on high memory config and I run some heavy process, it could've been combined with the ssh spam it went OOM. I still run all the things, without any issues now.