I was thinking of something which is purely data driven, connects to build, source control, code quality metrics reports etc and builds a picture of each developer on the team based on their contributions, breaks, fixes, # of languages worked on etc?
Do you think it's even possible to build a good picture of a developer's performance based on data driven metrics?
I would start measuring projects or teams first, getting a solid set of metrics for each and once you have that you can start tracking down individual developers.
Even so tracking developer productivity can be hard, a good chunk of our work happens in our head. Who is more valuable the developer that wrote thousands of lines of code or the one that changed one but fixed a critical bug in production?
I highly recommended this book http://shop.oreilly.com/product/0636920020134.do
I was hoping to gather metrics in the following dimensions:
1. Versatile (can work on multiple stacks and languages) 2. Team Player (watches out for the team and does what needs doing regardless if assigned to them) 3. Detail oriented and delivers quality code (doesn't break the build, good test coverage, adheres to formatting guidelines) 4. Productive 5. Communication skills (can and does communicate appropriately)
For many of these, we could gather proxy measures, i.e.
1. Versatile (in the last year, how many different file types did this person touch or work on. 2. Team Player (did they fix a build broken by someone else, how often do they fix their own broken builds)
etc etc
These kind of metrics never provide accurate information. You might as well just pick your "good" devs from a hat.
For example, I'm the main code reviewer for our front end solution but I haven't written a single line of code for it. Am I less versatile than a dev who does basic work across the entire project?
But I agree, it's all imperfect. I also don't think these metrics would be the only criteria by which someone is judged.
Currently I'm measuring:
- How many tickets have been closed by each developer (we use mantis)
- What is the (average, average of 5 best, average of 5 worst) time for solving tickets
- For each average from above I obtain number of lines changed and also calculate average (mantis connected with git)
So for given period of time I know:
- how many tickets were closed by each dev
- in average - best time, worst time and usual time for solving tickets
- in average - amount of lines of code for worst time tickets, best time tickets and usual time tickets
I then look if lines representing average score are closer to lower or upper averages.
However all above is just to provide some orientation, my team is still small enough so I know each developer myself and also I'm involved in solving most issues so I base on that knowledge.
Not all bugs/tickets are created equal. Just counting them would incentivize to cherrypick the easy ones.
Why do you need a hard number? If you work on a <10 person team then you should already have a good idea of how everyone is performing.
I worked at one place where our bonuses were based on tickets closed and it lead to so many problems. We would get support/qa to create multiple bugs for a single issue and we then would do different fixes for each one of them even though they all really stemmed from the same core issue. A lot of bugs came in over night so a couple of us would log into the bug tracker from home and assign ourselves all the easy bugs. Devs, when they got half way through a bug and realised it was going to be more difficult than they thought would just put it back, or leave it half finished. Etc. Be very careful what you wish for :)
And just little comment on your first suggestion - you need to have some data supporting your story when you go to see the financial exec and negotiate rises. Your subjective opinion may mean a lot to you, but in many cases may be not enough when you want to do money talk.
Hopefully one you are wise enough to keep to yourself :). I've seen so many problems even with the use of relatively sane metrics.
> what would you suggest to use to all those folks who are building new team?
Management, not metrics. At best you can use metrics as a rule of thumb to give you early indicators of possible issues.
For a new team though you need to work together with them for a while to learn about their performance and capabilities. There is no other way. Perhaps one guy in your team is thoughtful and methodical and takes a week per issue but can solve complex issues, and another girl in your team can fly through half a dozen simpler issues in a week. That's actually the kind of diversity you want to encourage and it's something you can easily destroy if you consider #tickets closed an important metric.
You should never mention metrics as a measure of performance with either your subordinates or your superiors. Your subordinates will work to the metrics rather than do what needs doing. And your superiors will start relying on the metrics instead of on your judgment.
For example if you collect and use the #tickets solved per month as a metric then if you need to downsize your manager will say lets get rid of the people solving less than x tickets per month. Oh wait that includes some of your best people! Now you have an uphill battle, especially if you have used the #tickets closed as a reason to promote someone in your team.
Edit: Basically the potential gains are minimal and the potential issues are numerous.
Dodgy metrics probably won't convince your exec either.
If you are consistently building your team members capabilities then convincing the exec is easy.
For example you can say Ben has solved complex cases A, B, and C in the last six months, and the work is clearly at a senior level but his pay doesn't reflect it. Lets fix it.
Or: John has taken on supervision and mentoring tasks for Sam, Ben, and Steve, while still performing at a high level. He has mentioned to me in our one on ones that he wants to take on more leadership tasks, and we have project Y coming up. I think we should put him in charge of the new project.
Etc etc. You don't need silly metrics.
I get that you want averages but even between senior and junior devs it seems like this would be massively weighted towards devs who don't pay attention or bother to look for a 'right' solution and instead go for the quickest
Is tickets closed a good metric?
Why is this? Well, it's because the amount of tickets, code, or even time spent "logged in" doesn't matter. The only thing that matters to a business is if they can deliver on time so the product can keep moving forward and others aren't waiting for them.