The conntrack(1) utility also has some fun brokenness. For example, there's a TOCTTOU issue when you flush connection state, as Calico and other k8s CNIs do regularly. The utility queries the kernel state, builds a list in user space, and then iteratively deletes each session. But if a session expires between when its reported and when the utility tries to delete it (for example, hitting the standard UDP session timeout, or a TCP FIN, which happens all the time), the delete fails with ENOENT and the utility immediately exits without flushing the rest of the state.

Calico and other controllers resolve this by calling conntrack(1) in a loop until success or a limit (e.g. 3, 5, or w'ever magic number) is reached. But even then you regularly get one more error than the limit, triggering alarms.

I submitted a patch last year that added a command-line switch for suppressing certain errors, but the netdev mailing-list is high traffic and I never got any feedback. I never expected my patch to be accepted on the first round; not even the approach. I can think of several other alternative ways to address the issue, such as not exiting on error until working through the queue, utilizing exit status codes for reporting the reason, etc. But it's unfortunate I got no feedback whatsoever considering it's a substantial pain point in the wild and a clear (IMO) defect in the implementation--because the error is inevitable and to be expected given the TOCTTOU race.

Of course, if I were writing a controller like Calico I probably wouldn't be shelling out to conntrack(1), but instead making the netlink calls in-process. But I'm not most people, especially in this particular case where everybody else manifestly treats conntrack(1) as the de facto programmatic interface.