"Caddy L4" aka "Project Conncept" might be what you're looking for:
https://github.com/mholt/caddy-l4
"Project Conncept is an experimental layer 4 app for Caddy. It facilitates composable handling of raw TCP/UDP connections based on properties of the connection or the beginning of the stream."