If the client is still at the same address it was last time it contacted the server and the server sends data to the client it will be transparently handled. This means that persistent keepalives should be used on both sides. It also still leaves open the possibility of the connection being lost if IP addresses on both sides change during the period between the keepalives or if there is any kind of network disruption preventing traffic from the server to the client during the period the server IP address changes. I had this become an issue for me a few times when I ran a wireguard server at home.
This won't work if the server is being blocked. My mobile ISP blocks incoming UDP traffic unless the phone establishes the "connection" first. Since change of the server's public IP means that "connection" is broken, all unrelated UDP packets never arrive.