Hacker News Reader

Comment by Blackstrat | Hacker News Reader

Companies still use it. The company I retired from three years ago still uses it. If the manager is doing their job, their team’s performance, relative to the rest of the company, is not normally distributed. Stack ranking is a stupid idea, as many of Welch’s ideas are now considered.

View on HN

Curve fitting "performance" to an "expected" bell curve is clearly idiotic, even from a mathematical perspective. In the described case one won't have a symetric bell curve and the population is going to be too small, even if measuring performance was done right, which it clearly wasn't. Whoever invented this process has done so without understanding normal distributions. It's a bunch of mumbo jumbo.

A quick Google shows estimates of about ~10K Blizzard employees. That's a plenty large enough population for mass-population trends to show up.

It may be, but remember that most companies do stack ranking at some roll up level, not over the entire population. I had 40+ developers in my organization with two peers having comparable sized teams. We were expected to horse trade amongst ourselves to insure the VP got his distribution. However, the VP would always try to browbeat each of us into matching the curve. It was pretty intense and he frankly didn’t give a rat’s rear end who got steamrolled in the process. To say I was thrilled to have outlasted his employment would be an understatement.

And in your imagination of stack ranking, every employee gets a rank number from 1 to 10,000?

That would be neat, but probably bad for a variety of other reasons. Like, why is all of sales getting fired? Well, they unfortunately all ranked poorly...

It's more like, a manager has 7 employees and he has to stack rank them 1 to 7 and manage #7 out and rehire a replacement. Was #7 a bad employee? Why would any system of management assume that they are? Such as system assumes that the sea is full of employees of rank #1 to #6 and that the difference between 6 and 7 was so great as to warrant the costs associated with termination, hiring, training, on-boarding, and ramping up a new #7. Will they be better enough to warrant doing it all again?

But the sample size could be a billion and it wouldn't matter if the process that generates the population isn't random.

If Blizzard was just hiring people off the street then yeah, sure I'd expect a bell curve.

But given the whole recruitment process, it's not necessarily going to form a bell curve.

Plus, the implementation is usually at the manager level - so each manager has to meet a quota of x% good, y% okay, z% bad etc., and each manager doesn't have 10k employees.

Sure. That's a large enough population that you're going to see a vaguely bell shaped curve, even if all the employees are exactly alike. From just random day to day variation. If you punish and reward based on that, you'll get better results, right?

W. Edwards Deming is rolling in his grave.

My company has about 4K employees. My team? 20. As best as I can tell, we have two stellar employees, two terrible, and 16 who are all basically the same. How these 16 are ranked is based on how well they get along with their boss, the PM that manages the projects they're assigned to, and the managers of other teams that they periodically interact with.

Yet somehow, management/HR can classify these 16 with enough certainty to decide that 8 deserve good compensation/promotions, and the other 8 should be PIP'd?

In some teams I work with, every single employee is what I consider an 8 through 10. Yet the 8's should be RIF'd/PIP'd?

Evaluating and compensating employees is tough enough without using voodoo like Stack Ranking.

My stats memory is rusty, but...

Doesn't a normal distribution typically arise during random sampling? I.e., if there's some statistic you calculate based on a sample, then when you take a bunch of independent population samples, that statistic will tend to be normally distributed over those samples.

If that's the only reason we'd typically expect to see a normal distribution, then I don't understand why we'd expect employee ratings to be normally distributed.

Exactly, they're not random samples, not normally distributed and the population is too low for a single team. It's all bs based on junk science. Lies, damned lies and "statistics".

They may be normally distributed or may not be. Actually hard to tell even given the data.

But you can make a plausible argument against the performance metric being normally distributed among your employees because that would mean the way you hired your employees was completely random with respect to that performance metric.

The way MSFT used to do it per team was way too small a sample even if it was randomly distributed.

But of course it isn't randomly distributed because you hope you recruitment process isn't just picking up random people from the street.

So yeah, it makes no sense - it's just cargo cult business school science.

Ignoring some techincalities, the normal distribuiton pops up when you add together a bunch of independent random values. So in 50 question exam, I might have a probability p of getting any question correct. My final score might be something like the sum of 50 biased coin tosses, which will be approximately normal.

I don't think there's any reason to believe that employee scores (out of 100, say) should be normally distributed.

Yes, you are describing the sampling distribution, which is the distribution of a statistic like the mean. This is different from a distribution of an actual sample. The confusion people encounter comes from the fact that statistical nomenclature couldn’t have been more confusingly designed if one tried.

Also, you have an HR department throwing random variables (while trying to hire the best) that's not going to help you getting a random sample.

Why is it distributed? Maybe if you took people at random but:

(sarcasm) shouldn't you have all good people because your leetcode hiring process only produces "the best" (/sarcasm)

Add to the fact that "rankings" are often largely political. There's a "minimum talent" aspect and then the last 1-2 points is most often political and representative of machiavellian power levels in the corporate dominance hierarchy.

Also, the article headline should explicitly call out Stack Ranking.

Stack Ranking basically admits one of two things:

1) your recruiting sucks

2) your work culture sucks

Well, and #3: corporate culture doesn't value balance or having any "bad times" in your life.

Reply on news.ycombinator.com