Hacker News Reader

Comment by vlovich123 | Hacker News Reader

The post looks reasonable to me. The point he's making is that no metric is useful for measuring developer efficacy/productivity & the underlying unstated reason is that dev works tends to be random; problem A is nothing like problem B & there's no way to objectively measure the quality of solution for problem A against another developer's solution.

He's instead proposing metrics that you can use as a high level barometer of project health. What outcome are you looking for? You can't mean the final shipping SW because that's usually way too late. You want to figure out if your velocity is going to let you hit your ship date well in advance of the actual ship date so that you have time to deploy process fixes/hire more people so that you do hit the ship date.

View on HN

> He's instead proposing metrics that you can use as a high level barometer of project health. What outcome are you looking for?

None of the proposed metrics capture project health. Test coverage is the closest and that still isn’t great. (Unit tests are important but you can have 100% test coverage and a 0% working product. In fact, this is how every project begins before the first line of code is written.)

No one gives a damn how many pull requests happen or how many days apart they are.

1) Does the software work yet?

1a) How much of it works?

2) Does it do what we want?

2a) .... everytime? Less? How many 9s?

Report on the things that matter. Track the things that matter. Everything else is actively harmful and distracts your teams from doing what matters.

Also, when you ship the software you haven’t finished, you’ve begun. It is absolutely still useful to keep metrics on outcomes and track improving them over time. It is absolutely not “too late” and any planning that assumes software is done after you ship it is flawed. Features can become done, software never is.

They're useful to measure in that _changes_ to them are indications that something might be happening. Use them as canaries.

For example, your velocity has changed. Why has it changed? Are you bringing new people up to speed? Have you suddenly hit some unexpected complexity? Your mean time to resolve issues has gone down. Are you releasing code with simple bugs you should have caught? Have your team started picking them up sooner?

Whenever something changes there's questions you can ask, the answers to which might well help you do better. Using metrics as an early indicator of change can help.

1) If you need metrics to answer those questions you have a larger problem than metrics can solve.

2) That's an argument for literally any possible metric. You could make an equally passionate argument that looking at and keeping track of free/busy times on calendars or soda consumption or how full the garbage cans are on the floor as an indication and early warning sign that something changed. It doesn't mean those are good things to track. Pay attention to the things that matter. Align whole teams and organizations around those things. Don't let them get distracted with metrics that don't matter.

3) Metrics are a double edged sword and influence behavior. If you're tracking the wrong things, you will get the wrong things.

People would like to have simple proxies for measurements instead of doing actual work. I totally agree all those metrics are useless and only real indicators are "is it working yet" and "are users finding it useful". Because I can make loads of pull requests that are perfect but are not working for end users, and have 80% test coverage for functionalities users never use.

Funny is that as project leader one should click through project as in using it to see if it is working instead of wasting time on finding proxies. If leader/manager does not know how to use system or never clicks through then he is useless...

Indeed. Plus, I would add that measuring the outcome (definitely a must-do, not arguing on this) is more about the product. The article was more focusing on metrics for the engineering organization, how it performs as a team. Could you say that your engineering organization is collaborating effectively, only based on the outcomes?

Yes. Divorcing teams from outcomes and their products is an anti-pattern. Focusing on metrics for an engineering organization not linked to the things that organization builds is harmful and will produce the wrong results.

Evaluating teams on outcomes will force them to collaborate to make something great. Evaluating teams on collaboration will just make teams do whatever it is you’re measuring.

Reply on news.ycombinator.com