The most interesting part of this is the k8s integration to find your peers and connect to them. I've wanted this for a variety of projects; I wonder if this is worth extracting into a separate library?
I'm not sure how it would be possible to do it in a "library" (without external centralized storage).