We use pingdom to measure external uptime, so network issues will count as downtime as well.
Do you have any other questions about methodology? I'd be happy to elaborate.
Do you have any other questions about methodology? I'd be happy to elaborate.
A great example is that as I write this, I just realized that our public Pingdom network status reports are offline... I count this as downtime against our global availability stats, but it isn't an event that would show up in my pingdom reports :)
For our application outages, it's been pretty well correlated that if pingdom can't get a 200 OK on the test pages we've set up, it's been down. And I don't think we've had much if anything slip in under a 200 OK but still being down.
I'm sure we're still off by a couple of minutes here and there, but the big picture should be quite accurate.
Our test logs in, causes some data to be fetched from the database, and renders a page which we then check against what it should return.
We haven’t (to date) had either any false positives or false negatives (when it alerts, the site is really down, and if it doesn’t alert, the site is really up).
This obviously isn’t a replacement for functional or integration tests to ensure that a commit doesn’t cause a piece of the app to stop working, but it does test the full infrastructure stack to make sure that it’s performing the way we expect it to.
I'm just trying to point out that the definition of "uptime" is hazy at best. :)