It looks like I missed the option to set an htpasswd file directly on the radicale server. After a second look at the Radicale documentation it looks like it supports both, htpasswd directly on the server or configured on the reverse proxy.