Try looking at options ServerAliveInterval and TCPKeepAlive, they should prevent SSH from detecting when there's temporary transport issues and persist when wireguard roams between IPs.