Hacker News Reader

Comment by mathattack | Hacker News Reader

Here's the obligatory Dilbert -> http://dilbert.com/strips/comic/1995-11-13/

This sounds like a cheating problem, not a measuring problem. The same things happens when people try to measure schools.

My sense is you can't give up on measuring things, because you can't scale an organization without metrics. You just have to balance them with understanding what is really going on. Or rather you need to have the metrics tell us "What's going on with the business" rather than "What's going on with the individual".

View on HN

The problem isn't measurement, it is tying rewards to achieving certain measurements. People will do what you reward them for doing, even if that isn't what you actually wanted them to do.

The real issue is that identifying the correct things to measure and reward is really, really hard.

I read once about a factory that was trying to improve output so they started rewarding shifts of workers for reaching production goals. But to achieve those goals workers started cutting corners and quality went down. So the company installed a device to measure the quality of the widgets, which worked for a while but then quality went down again. The workers had figured out they could just keep measuring the same in-spec widget over and over again to make the machine happy. Next the company altered the quality measurement machine so that it would issue an error if it did two identical measurements in a row. Again quality went up for a while until the workers figured out they could just rotate a couple of good parts through the machine to fool it.

I have a good book on the subject: Measuring and Managing Performance in Organizations.

The TL;DR version is that any job worth doing has some hard to measure components, so the best you will get using measures as incentives is to lag in the hard to measure parts. So the recommendation is not to stop measuring, but to make individual measurements invisible to management, and be just tools that people can use to improve their own performance. If your team is doing far worse than average in a measurement that you all find important, you will work on it anyway.

> The same things happens when people try to measure schools.

I think you have this a bit backwards - the cheating is actually the symptom of people trying to measure stuff in schools and then make those measures directly affect social status and future success of students. Cheating exists because the system doesn't reward you for learning the material, but for passing a test.

I was thinking more of the cases where the teachers and principals change the students test scores to pass state exams.

Here is one example of many -> http://www.cnn.com/2013/04/02/justice/georgia-cheating-scand...

You can't give up on measuring things, but you don't have to let people know what you are measuring or make the measure materially to them. That's there the trouble starts.

The challenge is then you lose accountability, and the ability for people to work on what they are weak at.

There's a good argument that we shouldn't attempt to work on our weaknesses but that we should work on our strengths instead. I don't have a link but I've seen reference to this in research about performance reviews.

I've seen similar. I kind of view it as "A couple As and a couple Cs are better than all Bs" The one caveat is that you don't want any Fs. For example, to be a great programmer, you don't need to be a great English writer, but if you can't communicate at all, your work will suffer. There's a minimum level after which it's better to focus on programming. If you're a great accountant, you don't need to be a master of spreadsheets, but you need to be good enough to get the job done.

How do you know if the soldiers you have assigned to a project are any good without testing them?

Test the project.

Measurement isn't the problem, it was the interpretation of the measurement.

They would apply the scores to the metrics used to determine suitability for promotion, probably because there isn't much you can measure to measure the performance of a guy who sits in a hole all day.

What metrics does an ant hill use?

I believe the current consensus is that KPIs are communicated via pheromones. Both the location and density of pheromones are meaningful to the decision-making process of an individual ant. The large-scale behavior of the colony is a result of the totality of those decisions.

You make an unfair comparison, however. Numerous factors that constrain ant behavior (brain size, genetics, sterility, energy requirements, etc.) simply do not obtain for an organization of unrelated human beings.

Reply on news.ycombinator.com