Matt, thanks - can you elaborate on your hack of the airflow script? What type of events do you set alerts on?
for usage -- I just stick a call to a function in bin/airflow before the cli is run:
https://gist.github.com/anonymous/7d2a50d863ff2210ac69563c76...
This was to mainly catch "DAG is deadlocked" type of errors that seem to only come out of the logs.