Hacker News Reader

I think there's something to be said about keeping some key metrics super simple so that "everybody" can understand without having to refer to a formula or arbitrarily set thresholds. I've been using 99 and 90 percentile avg performance. It captures enough information in most cases and doesn't require any explanation.

View on HN

Hi edouard,

I completely agree! Keeping toplevel metrics SIMPLE is the key. Of course, simple but also not misleading you into any wrong beliefs.

While we liked the 95 percentile approach, we decided against it. Its still too focused on the actual response time itself, which we thought was less relevant than the number of users experiencing bad performance.

I think for us the bottom line was:

A) If you are having a site-wide performance issues, 95% percentile is a good metric.

B) However, if you have more isolated issues (we find this happens more often to more mature sites), satisfaction score is better.

Best, Mike

Im seeing a lot of "averages are bad" etc but I think you come closest to what I had in mind: there isnt anything inherently wrong with using simple metrics. The caveat is you just need to keep in mind and understand their limitations and where they fall down. I think a lot of people understand that using 99 or 95 percentiles and what not but just failed to lay the reasoning out.

Reply on news.ycombinator.com