Triggering a DAG externally fails consistently for 1% of requests, and 100% of the time when the web server API goes down, which is too often (502). I understand that in the realm of Airflow, triggering a DAG externally is perhaps a bit of an anti-pattern, but it should not have been documented then:

https://cloud.google.com/composer/docs/how-to/using/triggeri...

I have cases open with the composer team, and I am upgrading the cluster again this weekend, but I have lost hope at this point; I am designing a solution to stop using external triggers.