It runs on a Linode instance with a webapp whose sole responsibility is to respond to Pingdom requests. There are two URLs that Pingdom looks for: one that returns a 500 if the JSON file indicates an issue that warrants texting me. A second that returns a 500 if the JSON file indicates an issue that warrants emailing me for a lower priority issue. Pingdom is configured accordingly.
If for any reason the JSON file has not been written in the past 10 minutes (?) or cannot be read and parsed, both URLs return a 500.
The script has a log file, so when I get an alert I can check the log file to determine what is wrong.
This is likely atypical, but it works really well for me. My scripts do the work of monitoring the heck out of everything. I only need Pingdom (or a service like it) to monitor two URLs and do the texting/emailing.
But my overall approach is to think of monitoring like unit tests or integration tests: when I think of something that could go wrong, I try to make sure there is monitoring that can detect it and alert me. When possible, before it becomes urgent. And when something does go wrong that is not automatically detected, it's a high priority to add monitoring around that.