Same Ingress, Different Defaults: What Changed When We Moved to HAProxy

We're a company providing hosting for SME in Switzerland and recently moved our managed services' ingress and our PaaS Deploio from ingress-nginx to haproxy-ingress (because ingress-nginx retired).

Problems came up with the defaults, not with the annotation mapping: 50 instead of 60 second timeouts (almost 10 times longer on exports that finished faster than a minute before), 2000 per pod for maxconn and unchanged HTTP/1.0 passing through to a rejecting S3 backend. And on musl, there was bcrypt basic auth behind a process-wide spinlock, pushing one pod past 2 CPU cores with 155 concurrent connections.

In the linked blogpost, you'll be able to read about the 8 differences and values we ended up with.