I'm pretty much in the same boat.

I've been experimenting with collectd+influxdb+grafana, and have been pretty impressed with how easy it's been to build a bunch of metrics.

The addition of Kapacitor seems promising- alerting was a piece of the puzzle I was missing. Some people were using Sensu, but the grapevine was giving it less than stellar reviews.