How does Caddy compare to nginx performance wise? I have a similar setup like you running on nginx.

For loadbalancers I mainly care about performance and not usability of the configuration language.