Following the e-mail analogy: Inevitably, there will be contacts of yours who didn't get or read your notification, or contacts of yours who aren't in your contacts list.
Start on a server, but your real identity is attached to a cryptographic key, not an e-mail-like identifier. That would allow you to move around, and maybe one day get rid of domain names altogether (using something like yggdrasil or tor to host and connect servers, for instance).
Signal offers no such choice.
Even if you don't do this, you can still reach contacts on the old server and middle through.
If you switch from walled garden to walled garden like WhatsApp to signal there is no migration path at all.