I see 3-4% of 1GB ram usage for caddy only. Note that in my case caddy serves as reverse proxy and there is also very little traffic.