Hacker News Reader

Comment by bunderbunder | Hacker News Reader

I regularly screw these scores up by honestly answering the question I was asked.

> On a scale of 0-10, how likely are you to recommend Netflix to a friend or colleague?

> You answered 0 to the last question. Why wouldn't you recommend Netflix?

It's 2018, people. Everyone I know either already uses Netflix, used to use Netflix, or doesn't own a computer. If I started going around making a recommendation like that, they would think I'm a prat.

View on HN

I think the methodology assumes that only a small fraction of people are pedantic enough to do that

Also the "Why did you give that rating" follow up question can help spot whether a significant number of people interpret the question differently. I've seen surveys for financial institutions where a lot of people write "I love my bank but I don't feel comfortable recommending financial products to people" which is another example of an individual's score not accurately representing their experience as a customer.

I gave an extreme example for comedic value.

The kind of situation I'd expect to see more often in real life is that NPS scores are inflated for fun luxury goods, because they naturally inspire more enthusiasm, and deflated for essentials, because very few people are even capable of getting excited about socks.

Mostly I was meaning to hint at, where I've seen things like NPS go off the rails is when people assume they measure what the marketing pitch says they measure. The reality is invariably more subtle.

I had similar call just after a 10 day stay for a kidney transplant they rang up to ask how likely I would be to recommend the hospital to someone!

The English Friends and Family test a large scale implementation of this kind of scoring system.

Data is available here: https://www.england.nhs.uk/fft/

There are a handful of really good sock companies out there right now which I regularly recommend:

* Darn Tough

* Stance

* Bombas

NPS +10

Recommendations are also skewed by the needs and interests of the pool of friends a customer has. I'd recommend (and brag about) a good computer keyboard or monitor to almost anybody I know, because 90% of my acquaintances spend a lot of time in front of a computer; I can make an educated guess about the small minorities who might be interested in a certain restaurant (location-constrained) or a board game or a bootleg CD; but I don't think I could "promote" to anybody things like old roleplaying games (I'm the most eager collector I know) or a graphic tablet (the handful interested people I know already have a good one).

And what about the buying potential of one's friends? Consider plants: students could help sell their student friends something inexpensive that fits in a small vase in a small dorm room, farmers could help sell other farmers enormous amounts of product.

I used to do that, then I realised that a minimum wage call centre employee's salary/bonus/happiness probably depends in some small way on my answer, so I stopped being clever and just gave the answer they're clearly asking for.

(I do keep an exception for NPS surveys on automated interactions. No, Barclaycard, I will absolutely not recommend you to friends based on the text message you sent me about my increased credit limit, are you literally insane?)

It’s a truly broken system. I always answer 10 for call center employees, because often they’re not empowered to actually fix anything, and answering anything else often hurts that person instead of shedding light on the broken system they’re stuck in.

The only time I answer less than 10 for a CS person is if they’re actually rude or not trying to be helpful. I much prefer the surveys that ask multiple specific questions that split the interaction between the rep and the outcome.

You are not screwing it up as much as you think. A zero rating has the same impact as a 6 on NPS.

0-6 counts as a-1

7-8 counts as a 0

9-10 counts as a +1

NPS is an average of these numbers so if every user gave a seven or eight, the NPS would be zero (which is a mediocre but not terrible NPS score)

https://cdn.business2community.com/wp-content/uploads/2016/0...

It's actually the percentage of (+1) minus the percentage of (-1). 7-8's are literally not being counted (as you said 0's). Imagine if 990/1000 give 7 or 8. Out of the remaining 10, 7 give a 9 and 3 any number below 6. The count would be: 70%-30%=40 (which would be a good outcome). But only 1% of everyone that participated ended up determining the actual NPS Score.

I don't think that's right, and the example given in the article seems to agree with me. The 7s&8s are indeed counted, indirectly: the score is the percent of all responses (including 7s&8s) that are promoters minus the percent of detractors.

So, using your example, 990/1000=99% of people are passive, 7/1000=0.7% are promoters, and 3/1000=0.3% are detractors, so the NPS score is 0.7-0.3, or 0.4 (not 40). Passives aren't a part of the last calculation, but they affect the percent values for promoters and detractors.

That’s actually a fair answer. The quantitative aspect of NPS is to project growth. And the text is for useful verbatims.

Not sure that's a terribly uncommon scenario. Maybe a little dramatic, but not uncommon. This whole idea of recommending needs a little bit more context to be valuable.

How about "Your friend has decided to ditch cable and try using a video streaming service. How likely are you to recommend Netflix?"

It's even more critical with services that have very specific use-cases. How about a dermatologist? I don't think many people would say they recommend this type of service without some context. So, "If someone you know develops a similar condition, how likely are you to recommend DermatologyCo?"

You're presenting a bit of an extreme, pedantic example, but I think you're right in a more general sense. There are plenty of things that I wouldn't recommend to others simply because they're niche or are just in a category of things that wouldn't come up in normal conversation (perhaps they're just mundane every-day things). I might be 100% satisfied with something, but that doesn't mean I'm going to go telling everyone (or even anyone) about it.

Reminds me of the internal surveys at my company, but I don't think we're given room to elaborate.

> How likely are you to refer a friend to a position at <company>?

The answer is 0. That doesn't mean I think it's a terrible place to work, it means that I don't know anyone who is both interested and qualified.

Reply on news.ycombinator.com