Comment by gedrap | Hacker News Reader

Comment by gedrap | Hacker News Reader

Parent Full thread

Even the 'good' metrics you listed are not that good and can be easily gamed if they are important enough, e.g. can influence salary, etc.

Happy customers is the only exception, but it's essentially impossible to attribute this to a team or an individual and is more of a company's health metric. You could just replace it with revenue.

>>> products shipped, features delivered per team

Will lead to releasing products that should never be released, or releasing too early, or creating a lot of features just to increase the counter.

>>> bugs fixed

Then why ship quality product in the first place, if it is more valuable to have easy bugs to fix later?

>>> documentation written

Can result in poorly written, or way too detailed documentation which will go out of date the next day.

>>> individual git commits

Some people commit more than a dozen of times a day and happily push them. Or you can create tons of commits that deliver very little value (e.g. adds a single comment), but increases the counter.

I understand that it is all well intended, it just seems like it creates a game rather than solves a problem. Some people are great at adapting to games and rules, some people - not so much.

Quantifying developer's work has been discussed a lot, and I am still largely skeptical about such low level metrics. Essentially, it motivates people to maximize the score, even if it means doing things that are only good for the score, instead of good for the company.

I think reactions like this tend to sidestep the issue, because orgs should never adopt any one metric in isolation, but instead try to use many different metrics as part of a larger holistic picture, both of the organization and of individuals.

If you measure how much stuff got done on paper within its agreed-upon deadline, sure, people may skimp on quality in order to ship something on time, and it probably would've been in the org's interest to delay release until all the work was really done.

And sure, if you measure against the number of bugs opened against production versions, trying to incentivize the least number of bugs opened as possible, you may invite people dealing with issues through back channels, so that they don't show up in the metrics.

And of course revenue is more of a company-wide metric.

But if you combine all of them - if your teams are hitting their feature development targets, and customers aren't complaining through established support lines, and revenues are up... is that not what makes a successful company?

In addition to your points, with which I agree, it also depends a lot on team culture. If you do your best to hire responsible adults and then treat your employees like adults most of them act like adults. When there is a healthy team culture of shared responsibility people who aren't pulling their weight tend to stick out.

The best way to avoid people gaming the system is to hire honest, hard working people who don't want to game the system. Then get out of the way and let them work.

You're right, of course, but hiring for culture fit is easier when you're a small company and you only have to hire for a few distinct roles. When you're a large company and you have a hiring quota of hundreds of engineers a year to deal with expected development requirements, well, that's where you start to need to "corporatize" and come up with good metrics.

I've never run a large company, or been high enough in management to try this, but I still think there is a better way:

In code we work in layers of abstraction. Large projects can be healthy when every layer is small enough to fit in your head.

In the same way, I think we could build a large company in layers of teams. Each layer should be manageable in the way discussed above.

Good modularity in code allows us to keep the whole problem in our head. Good team structure (requires good management) allows us to keep the management decisions at the tribe/human level that we are good at and not need the kind of mass, impersonal systems that lead to burdensome bureaucracy.

Of course, this kind of leadership and culture has to come from the top. It also requires that leadership explicitly avoid micromanagement, breaking the team boundaries. And requires continued commitment to treating your employees at every level like adults.

Of course, people are not code, but I think this organisational system of managing complexity can be applied to both.

It would be interesting to do some case studies, but I believe there are several large companies that have done this successfully, until top leadership changes (founder dies, etc.) and the new leader doesn't have the same trust of and commitment to employee/team autonomy. The dynamic changes, people notice and either leave or start gaming the system, etc.

There's some blatantly missing metrics here.

First off, there's no allowance made for mentoring and answering technical questions. If you go head-down and socially punish anyone who disturbs you - even for good reason - you'll get more done and get a raise. On the other hand, if you annoyingly ask questions that further your project at the expense of distracting people around you, that also shows up in your project metrics and not your coworkers'. This will likely end up resulting in a really dysfunctional, non-collaborative culture.

Second, technical debt. There's two casual factors here. First is that hacking together stuff helps your projects in the short run, while hurting the organization as a whole in the long run. Basically, another example of externalizing the costs of shipping your features. Also problematic is that working with lots of technical debt in the code base will stretch out your estimates, since there's tons of tech debt and making slow progress is the new normal. You'll "hit your feature development targets", but those targets will be a fraction of what they'd be with a better culture.

Third, and related to the above, is deadline inflation. Your organization ends up rewarding people who are better at convincing management to give them more time to ship things. Yeah.

1) Not so sure this is true. You may get more done individually in the short-term, but over the long-term, your anti-social behavior will cause the rest of your team to shut you out in turn, and your lack of visibility and understanding into what your team is working on will affect your future productivity and earnings. This is a Nash equilibrium at work, and good management will explain how the Nash equilibrium of co-operating on teamwork serves everyone's best individual long-term interests, even if the metrics naively seem to favor short-term action.

2) You don't need to adjust feature development metrics to get technical debt paid down. You set a target (e.g. 20% time to paying down debt), build a backlog of technical debt you know about, and either add the tasks to your sprint/version target (rewarding the team for completing them, like any other task) or come up with some other manner of getting them done, like Boy Scout code reviews (i.e. leave the codebase cleaner than you found it or fail review).

3) If you think you can ship better code by forcing arbitrary deadlines, then I hate to break it to you, but if the beatings continue, morale will not improve. And if you think you can substitute metrics and cash bonuses for good management skills to get people to work long hours so that you'll ship by the start of the holiday season (or other real deadline), then you don't understand how metrics is a tool for management and not a replacement for management.

I think I phrased my third point poorly, since I seem to have conveyed exactly the opposite effect that I intended. What I'm saying is that if you tie pay to performance against arbitrary deadlines, you're effectively paying engineers to pad their estimates. Estimation is definitely a pretty major game as it is, and we do ourselves no favors by paying people to play it more aggressively.

But all of that assumes that you're just plotting it on a graph and checking how it looks. Humans are capable of doing more assessment than that. Really a manager should do at least some cursory checks of documentation, etc. to ensure that it's not poorly written, etc.

At my last job we learned that productivity was measured with scrum points (so, estimates) done and your Github activity.

It was ... not really efficient.

I could see measuring the "points committed" : "points complete" ratio, but the points themselves...?

Reply on news.ycombinator.com