undefined | Better HN

0 pointsjonknee8y ago0 comments

Even Netflix finally moved over to up/down and they were famous for squeezing every drop out of their previous star based reviews [1]. In theory stars work better, but the issue seems to be everyone has a different ranking system.

For example, Uber seems to think anything but a 5/5 is a failure. I know this so I skew to accommodate, but in my personal ranking system I've only had a couple 5 star rides (someone really going above and beyond).

Up/down with an optional qualifier afterwards (e.g. "why were you unhappy?" after a thumbs down) seems to remove a lot of confusion.

[1] https://en.wikipedia.org/wiki/Netflix_Prize

0 comments

yoz-y8y ago

Maybe the problem is with stars and wording. Currently most of systems are worded (e.g. amazon) in a way that 3 stars is the base and people would add stars if their expectations were exceeded and remove them if they were not met. However at all places that I have seen it is like you say 5 stars is for expectations being met and it only goes downhill from that.

I think that a wording and iconography in 4 steps could be useful. -2 = something really bad happened, -1 = below expectations, 0 = happy customer, 1 = exceeded expectations. Forcing people to write a detail on any rating other than 0 would make most ratings 0. Angry people usually like to write comments anyways.

ghaff8y ago

I'm not sure an expectations-based rating system is the norm though.

To use an example I gave elsewhere. I order a cable from Amazon. It works. Therefore it met my expectations of a working cable. Yet, I think most people would interpret a 3-star rating as my being lukewarm on my purchase. I'm not. But what the heck do I expect a cable to do other than being a fair price and to work?

With respect to movies. Some movies get really built up and I go in expecting great things (e.g. Fury Road). I come out thinking they were just OK. So maybe 3 stars. But definitely not -1 or 2 stars. My personal expectations aren't necessarily a good baseline.

yoz-y8y ago

For a single data point they are useless. But I thing a rating in aggregate could be informative. Of course, the rating is self defeating as too many people who were positively surprised will raise the expectations.

For movies I would love to have reviews in style: Somebody writes up their expectations before going to the movie, and then rates the movie according to that. I find most "press and critics" reviews useless as if somebody is not a fan of story-less action movies, then why the hell is he reviewing them.

enobrev8y ago

This is how I see it as well. It would also likely get rid of comments / ratings like "I would give this 0 stars if I could". I think the numbering you mention is just as essential (especially 0).

The way I generally rate things falls in line with this idea:

5: Excellent

4: Pretty Good

3: Average; Unsurprising. Not Impressed, Not Disappointed

2: Kinda Sucks

1: Run Away

This definitely doesn't seem to be how everyone else is using the 5-star scale.

ghaff8y ago

This was (is? I haven't really used them much in a while) an issue with eBay's reputational system as well. Anything other than a Positive and "A++++++++++++ seller" was interpreted to mean they shipped you a box full of bricks rather than they took a week to ship things.

dredmorbius8y ago

Normalising a given rater's stars, or asssigning costs to higher / lower ratings, is another option. Essentially you have a "ratings budget" you can spend, up or down, on your assessments.

Stack Exchange has this to an extent, where negative ratings cost the rater points -- you have to really want to assign a negative.

1 more reply

stinkytaco8y ago

Everyone does have a different ranking system, but Netflix was good at predicting what I would give a movie, so my ranking system was the only one that mattered. Their special sauce in the background made it pretty accurate.

The problem with an up/down for me is that it doesn't capture an ambivalent reaction. That means transactions will tend toward the mode, imo. You will do enough to get a thumbs up, that's all.

kalleboo8y ago

I often wish it was an up/down/love

dennisgorelik8y ago

5-star ratings probably carry more information than binary thumbs up/down, but every 5-star vote is more complex to collect, so Netflix was probably getting less 5-star votes than they are getting up/down votes.

Overall number of votes matters too.

j / k navigate · click thread line to collapse

0 comments

yoz-y8y ago

ghaff8y ago

I'm not sure an expectations-based rating system is the norm though.

yoz-y8y ago

enobrev8y ago

This is how I see it as well. It would also likely get rid of comments / ratings like "I would give this 0 stars if I could". I think the numbering you mention is just as essential (especially 0).

The way I generally rate things falls in line with this idea:

5: Excellent

4: Pretty Good

3: Average; Unsurprising. Not Impressed, Not Disappointed

2: Kinda Sucks

1: Run Away

This definitely doesn't seem to be how everyone else is using the 5-star scale.

ghaff8y ago

dredmorbius8y ago

Normalising a given rater's stars, or asssigning costs to higher / lower ratings, is another option. Essentially you have a "ratings budget" you can spend, up or down, on your assessments.

Stack Exchange has this to an extent, where negative ratings cost the rater points -- you have to really want to assign a negative.

1 more reply

stinkytaco8y ago

The problem with an up/down for me is that it doesn't capture an ambivalent reaction. That means transactions will tend toward the mode, imo. You will do enough to get a thumbs up, that's all.

kalleboo8y ago

I often wish it was an up/down/love

dennisgorelik8y ago

Overall number of votes matters too.

j / k navigate · click thread line to collapse