> Does anyone know if there's significant pain in maintaining a docker image that compiles source code like Caddy on rebuild?

It's insignificant. Multi-stage Dockerfiles mean you can build from the Golang image, pull mholt/caddy, compile to a binary, then distribute the result in a tiny Alpine image.

I believe the maintainer of the current most popular Caddy image will be going source-compiled, too, so you might not even need to roll your own Dockerfile.