ah good tip, I don't care about session affinity or custom balancing algos so that works. I'd imagine running in GKE or AWS you would also avoid the DNAT / conntrack overhead as pods by default use a routable VPC IP instead of a magic CNI IP. Would have to test that though.
Quote from related issue:
The NGINX ingress controller does not uses Services to route traffic to the pods. Instead it uses the Endpoints API in order to bypass kube-proxy to allow NGINX features like session affinity and custom load balancing algorithms. It also removes some overhead, such as conntrack entries for iptables DNAT.