RPGDot Network    
   

 
 
Stonekeep
Display full image
Pic of the moment
More
pics from the gallery
 
 
Site Navigation

Main
   News
   Forums

Games
   Games Database
   Top 100
   Release List
   Support Files

Features
   Reviews
   Previews
   Interviews
   Editorials
   Diaries
   Misc

Download
   Gallery
   Music
   Screenshots
   Videos

Miscellaneous
   Staff Members
   Privacy Statement

FAQ
Members
Usergroups
The skewed Top 100 - What can we do?
  View previous topic :: View next topic
RPGDot Forums > RPGDot

Author Thread
aldreth
Eager Tradesman
Eager Tradesman




Joined: 09 Feb 2003
Posts: 26
Location: Washington
The skewed Top 100 - What can we do?
   

Well this isn't a rant or anything, just wanted to see if anyone had any ideas for the Games Top 100 List. The list is very inaccurate, quite simply put, due to fanboys voting repeatedly against/for games they like/dislike. This changes the ratings of the list tremendously, which makes the purpose of it entirely null.

There must be a better way to filter the fake reviews and thats what im hoping to get ideas for. I was thinking that maybe the rpgdot team could add a feature that would require you to write a thorough review when you rated a title. Maybe require a certain amount of characters per review minimum or something.
Post Sat Jan 17, 2004 12:12 am
 View user's profile
Lord Chambers
Eager Tradesman
Eager Tradesman




Joined: 22 Dec 2003
Posts: 29
   

What top 100 are you talking about?
Post Sat Jan 17, 2004 12:25 am
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

@L_C: Look over in the navigation menu to the left, under "Games".

@aldreth: I did a little research, and I didn't find any instances of multiple votes from the same person. I found a variety of times where games were given silly scores (mostly 1's to good games), but they call 'em opinions for a reason.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Sat Jan 17, 2004 12:53 am
 View user's profile
Gorath
Mostly Harmless
Mostly Harmless




Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany
   

I have to agree with aldreth. As it is now the list is heavily manipulated. I talked about this with Garrett some time ago.
The list system puts too much emphasis on low rankings. Just ask yourself how many votes you need to compensate a single '1 out of 10' vote.

An idea would be to ignore the highest and lowest 10 percent of the votes. Or start at 5 points since we´re in a time were everything below 7 is low.
_________________
Webmaster GothicDot
Post Sat Jan 17, 2004 7:06 am
 View user's profile
aldreth
Eager Tradesman
Eager Tradesman




Joined: 09 Feb 2003
Posts: 26
Location: Washington
   

Thats a great idea Gorath, your idea would average the votes out alot better, but I think we would still have people with 1 word reviews and a score of 5 then


Yea dteowner the review's that I was primarily referring to were the ones that had like one word and a rating of 1. Those reviews are obviously not thought out and give the people who actually think of buying the game nothing to base the bad review off of. It's just to easy to switch ip's and reload, then repost under a different name with a 1 word review of nothing. I really hope you guys can do something about it
_________________
`''''''
(0 0)
-(_)--oOO---------
Work to become,
not to acquire.
-----------oOO-----
__|__|
|| ||
oO oO


Last edited by aldreth on Sat Jan 17, 2004 11:41 pm; edited 1 time in total
Post Sat Jan 17, 2004 12:23 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

quote:
Originally posted by aldreth
It's just to easy to switch ip's and reload, then repost under a different name with a 1 word review of nothing.
That seems like an awful lot of work just to trash a game on a minor ranking system. Do you really think there's that many people with that much time on their hands that also have the compulsion to childishly wreck a ratings system?

Seems to me that there will always be a way for someone to post a ridiculous score, regardless of what we do. The one thing I might add would be some guidance about what the scores actually should mean. I'm disappointed with the number of reviews that list a variety of "dislikes" and then award the game a "10". Like Gorath said, these days anything below a 7 is only awarded to complete bugfests, but that's partially because there's no instructions explaining the requirements of various scores. Such a legend will do nothing to slow down abuse, but might make the ratings a little more meaningful, if that's the goal.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Sat Jan 17, 2004 4:26 pm
 View user's profile
aldreth
Eager Tradesman
Eager Tradesman




Joined: 09 Feb 2003
Posts: 26
Location: Washington
   

Well I realize that people look at the ratings page for differing reasons. I primarily look at the reviews for games that I am interested in, to get good feel of what to expect should the reviews sway me in a direction to actually purchase the title. I really like to read the user reviews to get the opinion of true rpg fans, which rpgdot's frequenters are among the best.

I do think there are alot of fanboys who destroy the rankings system and for reasons unbenounced to me, have the time and take the time to actually do it.

i was also going to ask why we don't have to login to post a review, as it is right now, you don't even need to be a member of rpgdot to post a review.

_________________
`''''''
(0 0)
-(_)--oOO---------
Work to become,
not to acquire.
-----------oOO-----
__|__|
|| ||
oO oO


Last edited by aldreth on Sat Jan 17, 2004 11:29 pm; edited 1 time in total
Post Sat Jan 17, 2004 10:34 pm
 View user's profile
Ariel
Harmonious Angel
Harmonious Angel




Joined: 21 Jul 2001
Posts: 432
Location: Germany
   

quote:
Originally posted by Gorath
An idea would be to ignore the highest and lowest 10 percent of the votes. Or start at 5 points since we´re in a time were everything below 7 is low.

I find this idea rather compelling. What if we compressed the range of 1-10 to a range of 5.5-10 (for better granularity we would have to introduce half points).

To make it clear:
code:
1 -> 5.5
2 -> 6
3 -> 6.5
4 -> 7
5 -> 7.5
6 -> 8
7 -> 8.5
8 -> 9
9 -> 9.5
10 -> 10




I can imagine this to work out quite nicely. Myrthos could let a script run over everything to convert the existing scores. They would still be skewed, so this does not really take care of the root of the problem, but it wouldn't look as bad..

Aside from that, I'm all for a minimum word count when posting reviews.
_________________
“Through the sounds of falling rain, through the clouds of bitter times
I see the pure grace of your smile, in dreams of the warmth in your eyes”
- Tim North
Post Sat Jan 17, 2004 10:48 pm
 View user's profile
Lintra
Elf Friend
Elf Friend




Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES
   

quote:
Originally posted by Ariel
...

Aside from that, I'm all for a minimum word count when posting reviews.


Great idea! Then I don't think you'd need the skewed scoring system you proposed. I seriously doubt that the system would be gamed if you had to write a unique paragraph about each game.

And best of all it is a *simple* ideas ... those are generally the best (and the hardest to come up with!!).
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless=
Post Sat Jan 17, 2004 11:16 pm
 View user's profile
aldreth
Eager Tradesman
Eager Tradesman




Joined: 09 Feb 2003
Posts: 26
Location: Washington
   

I must say that Ariel and Gorath have great ideas. Ariel's idea would definitely work in keeping the past reviews yet still make the list better. Working with a script to convert the scores is a great idea.

I think adding anything that makes reviewing more in depth could only help. But I have no idea how much work that would entail for you guys

Would it take alot of work to add a minimum word count to the reviewers page? What about registered user's only being able to review, any negatives to that? Could it be implemented easily?

If you search through the Arx Fatalis reviews you find that the user "cmckj50" has posted 3 different reviews for it... How is that possible?
_________________
`''''''
(0 0)
-(_)--oOO---------
Work to become,
not to acquire.
-----------oOO-----
__|__|
|| ||
oO oO
Post Sun Jan 18, 2004 12:24 am
 View user's profile
Myrthos
Spoiler of All Fun
Spoiler of All Fun




Joined: 07 Jul 2001
Posts: 1926
Location: Holland
   

A game needs 10 votes to be even listed in the top 100. Regardless of the system there will be always people wo consider it fun to wreck it.
Even if we start at 5.5, what would keep anybody from using that 5.5 as a vote instead of the 1 they use now? The only thing that improves is that the average score is higher. It still has the same fanboys and gamehaters that vote a 10 and a 5.5.
How many words should one enter in the review in order for it to be valid? And should I also check semantically? Suppose there are 10 words needed. "One two three four five six seven eight nine ten" are 10 words also.

These options might improve things a bit but doesn't solve the problem. The only thing that solves it if there is someone who monitors every entry and then deletes the ones that are considered to be too fanboyish or the opposite.
I'll gladly forward an e-mail to the teammember who is ready to do this task every time a vote is being made so that he/she can monitor things.
_________________
Kewl quotes:
I often have an odd sense of humor - Roach
Why quote somebody else, think of something yourself. - XeroX
...you won't have to unbookmark this site, we'll unbookmark you. - Val
Reports Myrthos for making me scared and humbled at the mere sight of his name - kayla
Post Sun Jan 18, 2004 12:09 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

It will have to be several months from now, but I'll volunteer to tackle that project. My free time right now is, and must continue to be, dedicated to my job search. Once that is completed, I'll jump on that tiger.

The question would be whether it would be time well spent to slog thru the existing reviews. It might make more sense to chuck it all and install the new system. The downside of that, obviously, is that a lot of valid info gets tossed with the garbage. Something to ponder, and we've got a while to ponder it.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Sun Jan 18, 2004 4:53 pm
 View user's profile
Val
Risen From Ashes
Risen From Ashes




Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA
   

*Val remembers looking through all of the votes once to pick out the bad stuff. Val hides.*
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=
Post Sun Jan 18, 2004 5:06 pm
 View user's profile
Lintra
Elf Friend
Elf Friend




Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES
   

As to the word count, could a script be written to search the data base and do the count?
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless=
Post Sun Jan 18, 2004 5:42 pm
 View user's profile
Lord_Brownie
High Emperor
High Emperor




Joined: 16 Feb 2003
Posts: 575
Location: Unfashionable arm of the spiral galaxy
   

Any word count solution would be underminded rather quickly, as Myrthos has already mentioned. If someone personaly undertakes to read and consider everything that is posted, they are going to have a long few months. Even if all the old posts are disregarded, the initial wave of new posts by folks like me who have too much to say, about too many games, and with too much free time to say it, are going to flood the list
LB
Post Sun Jan 18, 2004 9:17 pm
 View user's profile


Goto page 1, 2, 3, 4, 5, 6, 7, 8, 9  Next
All times are GMT.
The time now is Tue Apr 09, 2019 8:46 pm



Powered by phpBB © 2001 phpBB Group
 
 
 
All original content of this site is copyrighted by RPGWatch. Copying or reproducing of any part of this site is strictly prohibited. Taking anything from this site without authorisation will be considered stealing and we'll be forced to visit you and jump on your legs until you give it back.