I think of this whenever people suggest that a star rating system is ideal.
I don't even rate AirBNB hosts (for instance) because I know that a good enough host will be punished. I only rate the top and the bottom because those are the only two ratings that have meaning any more.
I'm not sure how it works for either company - is it (transactions/rating score) or (ratings/rating score)? Downthread, we learn that Lyft gives null ratings an automatic 5.
https://www.businessinsider.com/why-netflix-replaced-its-5-s...
Plenty of people see 3 stars = Average, 5 = Amazing, but companies have decided that 3 = Bad, 5 = Acceptable.
I think it started to go wrong on eBay back in 2000. People gave any seller overly enthusiastic comments on any sale that somehow made its way to their door, as if that was something special and totally unexpected. And the sellers gave best ratings to buyers in return. Soon, both expected the other to provide a perfect rating for any transaction.
So perhaps the average for a review score/rating has just steadily crept upwards over time, til the scale collapsed into a two choice system with 'bad' and 'great' being the only options.
The median rating on Steam is about 80% positive (meaning 80% of players who rated it would recommend it). A game with a 70% Steam rating tends to be mediocre.
It's interesting that even though a rating system where you choose between "recommended" and "not recommended" is much less subjective than trying to rate a game on a scale from 0-100, in aggregate, you get similar scores.
Does Uber actually give instructions on how to rate? Does it say a normal, acceptable trip should be 3 or 5? What would a really great trip be anyway, perhaps if the driver says "hey, I just won a lottery and I'm celebrating with my passengers, take $10k", or perhaps something sex related?
Once Uber kicks out all the below-average passengers / drivers, will they take into account that the average will be pushed up for those remaining, so they will need to do further rounds of kicking?
I can't check myself, I've never used Uber. I've been unable to sign up due to some problem where my "email address is already registered", which apparently can't be fixed. I could just use a different email, but I don't really want to deal with a company that has such issues.
I’ve had a perfect 5.0 rating over 500 rides but would occasionally get lower ratings or 1-star for reasons I didn’t know. Bad smell outside comes through the vent? May a strong short stop in ridiculous traffic?
The ratings were meaningless to me, there’s no actionable result you can make. It just tormented me with some anonymous, ambiguous criticism of a already super stressful job that paid $20/hour.
And I agree with you -- this sort of disparity is why ranking systems like that aren't terribly useful. Take Amazon, for instance. People there seem to weight their ratings according to price, so something that is cheap and mediocre can get 4 or 5 stars.
All of this sort of nonsense taught me to ignore ratings.
How people like me give ratings.
5/5 - No problems 4/5 - Minor problems 3/5 - Major problems satisfied 2/5 - Major problems unsatisfied 1/5 - Horrible
How average ratings are interpreted:
4-5: Excellent 3.5-4: Okay <3.5:Terrible
This pretty much mirrors how my grades were given in school.
But how I actually give ratings is 5/5 regardless of problems as long as the service was delivered and 4/5 if not because I have anxiety. I couldn't bare to get a bad rating at my job after a mistake so I don't do it to others.
"A ride happened." -- 5 stars
"I don't think this driver should be driving anyone anymore." -- 1 star.
I've also reluctantly started giving minor tips in the app depending on whether my own rating is trending down or not, since drivers can and do rate riders lower for not tipping. (They have to rate before they take another ride, but they can retroactively rerate.)
There are now apps and other point-of-sale interfaces which default to 20% for "tips". Nice way to lower labor costs by outsourcing a voluntary fee to your customers, but this mostly destroys the signal/feedback value of the tip and rating.
Do the nudge crowd know they are engaging in semantic pollution, e.g. diluting the value of the data they are spending millions to collect, process and analyze.
Perhaps more importantly, do they care?
I think it's related to school grading systems. anything less than an 'A' means you did something wrong.
I remember when I had my first professional performance review, my manager asked us all to fill out the form ourselves and then discuss the difference between our ratings and his rating of us during the review itself. fresh out of undergrad, I gave myself mostly 5/5s and a couple 4/5s where I thought I needed improvement. I was momentarily shocked when I saw he had given me mostly 3s and a few 4s. he chuckled and explained that the company had very high expectations for software devs and that I wouldn't be getting any 5/5s in my first six months on the job unless I happened to be steve wozniak in the body of a 22 year old.
It's the same with services like Uber driving. There's always room for improvement, I expect the driver to make some "mistakes". Not the kind of life-threatening mistakes, of course, but more like "he could have been 30 seconds faster if he hadn't driven so sluggish" or "she may have treated my luggage with a little more care". And stuff like that is perfectly acceptable, at least as long as it doesn't all happen in the same ride. But a perfect, 5/5 ride must not contain any aspect at all that could have been improved by the driver in order to justify it's rating, and I haven't seen many of those.
I feel like this standard is very different from my experience with professors that actually expected you to get things wrong and compensated by making tests where there were, say, 130 marks available and you needed to take 93 for an A.
It feels weird to me that a 5/5 is for you means "couldn't be improved" because for me it just means "I got from point A to point B with my body and possessions undamaged."
Of all of the very surprising things I've learned in this thread, this is perhaps the most surprising to me.
In my view, getting less than an "A" doesn't mean you failed to meet expectations. Getting less than a "C" means that. Getting an "A" means that you performed at an exceptional level.
But there's clearly a generational shift involved in this that I hadn't noticed before now, and I think it's clearly for the worse. If "A" means "Pass" and everything else means "Fail", then there's no reason to have grades at all. Everything should just be pass/fail.
This is a really strange concept to me. So wherever you went to school, most students were always able to get 93% or more points in tests, so they "met expectations"? And there was almost nobody who managed to exceed expectations, so it wasn't necessary to keep some of the grades reserved for these people?
I can give some insight in the situation in Germany, where I live. We don't have grades A-F, but 1-6, with A=1, B=2 and so on. So it's basically the same system, just with numbers. But: there are textual expressions as well for each of the numbers. The one for the grade 4, or 'D', is "ausreichend", which means "sufficient" in German, and in the long explanation it is described as being the grade someone gets if he just managed to meet the expectations. So if you got anything above that grade, it means you were above the expectations, and if you were below, you failed to meet expectations. Typically, tests were designed such that 50% correct answers give the grade 4 and are thus expected from each student. The other 50% should be hard enough to get right so that getting a significant share of them right proves that you managed to exceed the expectations.
I guess this design should encourage students to improve themselves and to push themselves beyond the minimum expectations of teachers. By setting aside the largest part of the rating scale (1-4) to honor greatness and using just the smaller part (4-6) to punish laziness, people should rather be motivated by positive feedback instead of by punishment, which is clearly negative feedback.
Wow, I never would have expected that. I would only give myself (or expect) a 5/5 if I was really awesome in that category. On a 1-5 scale, the majority of my self-ratings would probably be 3s, with 2s for areas where I know I'm weak, and 4s for areas where I know I'm strong.
Ratings of 1 and 5 are, to me, reserved for the very end of each extreme. 1 means "nothing could be worse", and 5 means "nothing could be better".
it's hard to understand what a rating out of five actually means unless someone takes the time to explain it in plain english.
In my view, that is an unconscionable level of unprofessionalism. I'd rate any professor who told me that lower just because they told me that -- and say why.
If my manager wants to give me a lower score, that's not something I'm offended by; though I fully expect them to explain - in detail - all my shortcomings that resulted in their determination. I'll push back on just about all of their examples, even if I actually agree with their assessment.
I want my performance reviews to be as objective as possible. In my opinion, others should feel as though they have to convince me that I'm an imperfect employee, because they do.
Realistically, what does one have to gain by not defending themselves?
If I were the one performing employee evaluations, this would be a big red flag to me (unless you really were exceptional in all areas). It would indicate to me that you lack an awareness of your actual performance, and lacking that is significant. If someone can't reasonably evaluate how they're doing, then they can't reasonably be expected to be able to improve themselves in the areas where improvement is desired.
But that makes it impossible to indicate when something is great.
[1] https://qualifications.pearson.com/content/dam/pdf/Support/G...
[2] https://www.aqa.org.uk/about-us/what-we-do/policy/gcse-and-a...
If that's the actual game that Uber & Lyft expect people to be playing (as it appears that it is), then screw that. I'm not going to be using the services at all.
If it is, it's not difficult to introduce a system to deal with it (and they may already have such a system). All it'd need to do is normalize ratings or at least reduce the weight of negative ratings if the rater always gives negative ratings.
You don't express derivation in the positive direction because delivering a product or service as described without issue is the highest aspiration. It feels wrong to make anyone dance for their supper.
Oh, and btw: it's the same the other way round. I also sometimes put much more effort than strictly required into my job for a particular client, and I love it when that is recognized and honored. If it is, I am more inclined to do it again later. If not...well, not so much, then it's probably back to "standard service, just enough to fulfill the contract".
It's very disturbing to me that the highest aspiration is to just "meet expectations". That seems like motivating everything toward mediocrity.
My highest aspiration is to exceed expectations.
Like why even set expectations in the first place if your bar for perfect is "impress me"? Why is "I did everything you asked" less than perfect?
Many apps, including Uber, now also give the option to commend the driver for things they did well (e.g. cool vehicle, smooth driving, great conversation), which is also an option to express specific positive feedback.
"If your rating over the most recent 100 trips is below a 4.6, your profile may be at risk of deactivation."
It makes you a jerk, nothing more really.
Since I had no idea that 3 stars was a bad rating that hurt drivers until I read the comments here, I think that your condescension is very misplaced.
>It is "right" to provide an honest rating. What you seem to be saying is that it is the rider's duty to provide dishonest ratings.
That seems to me a staunch defense of doing what you want to do regardless of how it affects the driver.
From reading the comments here, though, I think I know what I'll do: opt out of the whole thing and avoid ride-sharing services in the future.
EDIT: I mean the royal "you", not you specifically. Also, I have been the a'hole before :|
I agree, in a sense. If you're working with a rating system that is inherently dishonest (as I just learned this one is) by redefining what ratings mean, then it's dishonest to supply a rating that you know means something other than what it should.
I was not advocating doing that.
If the company is using them dishonestly then inflated ratings are just serving the dishonest purpose.
Moreover, in the honest case (which I do not think applies) if the company is trying to use (known to be highly culturally variable) star ratings as a substitute for a simple binary “you would want to ride again with this driver” flag, they are idiots and overcomplicating what should be simple.
However, Uber doesn't value star-ratings the way she does, so she is wrong.
It's a car ride, not a movie. A perfect ride... is still just a ride in a car. If they get you there without incident and don't annoy you too much that seems like a 5/5.
Here's how the scale I instinctively would use would work: Getting me there without incident and without being too annoying would be a 3. If they don't annoy me at all, that would be a 4. If they do anything that makes my trip better (a better route, a good, relevant suggestion for where to go for something of interest to me, or just making my day a bit brighter), that would be a 5.
That guy probably deserved a "true" 5-star rating.
Direct action needs to actually affect the people who have the power to make changes. Stubbornly rating drivers 3/5 for a good ride just makes their lives harder, and let's be clear, nobody who is working at Uber gives a shit about that.
If people provided informative ratings, the cutoff would be adjusted. Uber's policies can't make the rating system worthless; the raters are doing it.
I stand by exactly what I said before: Uber is only able to set the cutoff it does because of rider behavior, and if the rider behavior were different, Uber would adjust to it, not the other way around. The riders have all of the power to determine what ratings mean, and Uber has none of that power, because the riders are the ones who set the ratings. No one at Uber is enough of an idiot to deactivate every driver on the platform because none of them reached a number that Uber made up. Or even to deactivate 70% of drivers because only 30% got perfect ratings.
Uber had lots of other choices, too.
2. I don't want the driver to do anything "special", and I can't even imagine what that would be. Balloons? Played my favorite song on the radio? Just get me from here to there, I'm looking for transportation not an "experience".
there should be room to acknowledge people that go above and beyond the requirements of their job without being punitive towards the people who do a fine but unremarkable job. I tip most drivers who don't completely fuck up the transportation part of the job, and I tip extra for really good rides, but I wish there were a way to recognize this in the ratings too.
Even if the average is rising, the average is still the average. Only people far above average are "exceptional", even if the average person of today would be considered "exceptional" 100 years ago.
The normal distribution curve may (or may not) be sliding along the scale, but its shape doesn't change.
It's a difficult UX and communication problem, I'll grant. But I saw several unforced errors on their part, like (from memory) marketing copy about "Uber only allows exceptional drivers on our platform". Um, no dude. That sounds nice, but it doesn't lead to usable ratings.
Is it?
It seems to me that they could just use imoji-like images instead of stars, ranging from Angry for the lowest rating to Ecstatic for the highest. The 3 star position would be neutral.