Site News: Rating System is Flawed.

What I mean is this: Suppose there are two posts. Post A has 10 votes (nine 5 star ratings and 1 3 star) and Post B has 1 vote (5 stars). On a Top Rated list, Post B will show up first.

This assumes that a rating is a cumulative score, like a grade (i.e. 9 answers right, 1 partial credit). However, when you are aggregating votes from a diverse user base this model doesn’t work. The more votes that something has, the more likely it is that there will be one or more contrarians or detractors–people who would have ignored the post if it had a low post count but are encouraged to vote against it because of the number of “views”.

In other words, a boring, non-controversial and overlooked post can rank higher then a much viewed, exciting post that merely has 1 detractor, even out of 10, 100 or more votes. Even oscar winners have a couple of negative reviews, that doesn’t make them worse movies than a home video whose only (positive) review is from the director’s mother.

I’m hoping that there is some way to use the total score of the ratings variable to find “true” highly rated posts. The only problem is, if Post C is rated 1 1000 times and post D is rated 5 100 times, Post C will be ranked higher.

I’m developing the fear that I’ll have to go in and do something along the lines of “1=-2, 2=-1, 3=0, 4=1, 5=2″ and then calculate totals. But I’m hoping that isn’t the only option.

As a temporary fix I’ve also included links to the top 5 most rated.

Once I figure this out, a TOP 100 page will be put up.

Also, a new thread was started on SomethingAwful.com. If you choose to contribute via SA, please use the new thread.

System(s): Site News

No comments

Rate this video:

 Loading ...

Related posts: