Basically every server program ever supports the notion of "which IP address am I supposed to bind to?" Normally the answer is either "localhost" or "whatever my LAN or WAN address is, for a particular interface". And specifying this (both for servers and for clients) is much less effort than replacing all the standard-ish ports with ad-hoc ones.
Just assign multiple IPv6 to your loopback interface and everything will work as expected. You can use either private IPv6 or even link-local IPv6 range.
There’s no need to waste thousands of addresss on this use case.
Yeah would hate to run out
As opposed to IPv4, where everything Just Works™.
Because parent is right, having multiple loopback addresses is just so nice for a lot of things.