Changing a port and enabling aslr are not "a lot of effort".
Are you familiar with the Swiss cheese model of risk management[0]? Obscurity is just another slice of Swiss cheese. It's not your only security measure. You still use all the other measures.