Anecdotally, changing the ssh port on a very low-budget VPS is worth the effort because the CPU time eaten by responding to the ssh bots can be noticable.
To be honest I don't think the people controlling those bots want to deal with us that makes it harder for them to gain access. Instead why not happily hammer away everyone's else port 22 with the bare minimum configuration? Those who enhance the security were never the targeted audience to begin with.
This is pretty insightful. Statistically, attackers are probably mostly looking for badly configured machines which are easy to exploit rather than hardened systems that take a long time to penetrate.
State actors and obsessed attackers are different, of course. But statistically even taking care of using the simplest precautions keeps one out of the reach of the broad majority of such attacks.
Do VPS providers offer some sort of similar firewall service outside your instance?
If you don't know that you'll be able to arrive from an IP or subnet - another option would be port knocking. (eg: knockd). Although, I'd try to avoid adding more code and logic to the mix - that goes for both fail2ban and knockd.
[1] ed: Note, the rationale for this is sound: the firewall (pf or nftables) is very good at filtering on IP - so better avoid introducing another layer of software that does the same thing.