Comment by pak | Hacker News Reader

Comment by pak | Hacker News Reader

"the average error was a staggering 24 minutes"-- The average is a terrible measure of centrality for data that is unbounded at one end, because one outlier (the 42 year off clock) skews the measure disproportionately; the median would be much more appropriate to report. That's unusually sensationalist number crunching for The Economist.

I'll bet that out of the clocks that doctors actually use to decide on treatment (tallying every device is a bit unfair; if a blood pressure monitor on some cart in a closet has the wrong time of day who cares) 80% of them are within a few minutes of the right time. And at the end of the day, doctors are supposed to be able to reconcile conflicting information, it's part of their training. This is a little bit like tallying up every textbook lying around a high school and getting concerned that the average book is three editions out of date and has 142.8 errors--calm down, most of those are not in active use and have some faith in teachers to check their material.

The research tested 1700 devices. A single device that is off by 42 years (reset to Unix epoch?) would have skewed the average by over 9 days. So unless a number of other devices had similarly extreme errors in the opposite direction (and even if they did, I'm pretty sure the researcher would have only looked at the absolute magnitude of the error, not their direction), I think it would be safe to assume that extreme cases were excluded from the average.

Bnd you're right, 80% of them were within 30 minutes of the right time.

> "One in five were off by more than 30 minutes"

In fact, given that the article mentions daylight saving time as a major problem, I'm not surprised that the average error approaches half an hour. At any given time, a lot of machines would be running in the wrong timezone.

Good point about the stats.

doctors are supposed to be able to reconcile conflicting information

They're supposed to wash their hands, too. (http://boingboing.net/2010/03/09/study-says-us-doctor.html) As a guy who makes dozens of mistakes a day, the ones I know about because a computer catches them, I'm really not reassured by the idea that human judgement makes up for machine error.

Average is a fine measure. You've inferred that they used a mean average. The article doesn't say what kind of average they took.

Let's assume they did take a mean. If the mean error for 1700 devices was 24 minutes, the total error is 40,800 minutes. That's 28 days.

Therefore, they didn't include the outlier of 42 years in their average. It's most likely a median (or mode) average.

It's pretty unlikely to be a median; if 20% of machines are 30 minutes off and 24 minutes is a median, then 30% of machines are between 24-30 minutes off, which would seem to be a strangely narrow band.

More likely, the 24 minutes is a mean average after excluding outliers. One of the standard ways to compute outliers is to measure the inter-quartile range---the "distance" between the 25th percentile value and the 75th percentile value---and call everything that is more than 1.5*IQR past the 75th percentile an outlier. You can do this without computing a mean first, and you can use it to exclude values before computing a modified mean.

Time errors between ten minutes and an hour are probably the most dangerous, so most devices being less than half an hour off is barely a comfort.

Reply on news.ycombinator.com