Basically this, yeah. You can add or take down backends without having to keep web server config in sync and reload it. (Though config reloads are pretty light anyways.)
I wonder whether it's technically generally feasible (possibly depends on net/http2's capabilities) to handover the connection from the old to the new server instance, comparable to what systemd does in terms of FD passing for socket-activated services.
To be clear, the new config is running at the same time as the old is still tearing down; all new connections get handled by the new config. Listeners are shared and transferred between servers.
We're all ears, though, if someone with deep knowledge of Go's internals can point us in the right direction.