We use graphite and cabotapp for metrics collection, aggregation and alerting