You will need to monitor: - ping - latency - temperatures - cpu utilization - ram utilization - disk utilization - disk health - context switches - IP addresses assigned and reaching expected MACs - appropriate ports open and listening - appropriate responses - time to execute queries - processes running - process health - at least something for every bit of infrastructure
once you collect that information, you need to record it, graph it, display it, recognize non-normal conditions, alert on those, page for appropriate alerts, and figure out who answers the pagers and when.