RPGDot Network    
   

 
 
Neocron
Display full image
Pic of the moment
More
pics from the gallery
 
 
Site Navigation

Main
   News
   Forums

Games
   Games Database
   Top 100
   Release List
   Support Files

Features
   Reviews
   Previews
   Interviews
   Editorials
   Diaries
   Misc

Download
   Gallery
   Music
   Screenshots
   Videos

Miscellaneous
   Staff Members
   Privacy Statement

FAQ
Members
Usergroups
The skewed Top 100 - What can we do?
  View previous topic :: View next topic
RPGDot Forums > RPGDot

Author Thread
Ariel
Harmonious Angel
Harmonious Angel




Joined: 21 Jul 2001
Posts: 432
Location: Germany
   

quote:
Originally posted by Myrthos
Even if we start at 5.5, what would keep anybody from using that 5.5 as a vote instead of the 1 they use now?

Nothing, but it would at least decrease the subjective skew, because as Gorath put it, everything below 7 is low.

quote:
How many words should one enter in the review in order for it to be valid? And should I also check semantically? Suppose there are 10 words needed. "One two three four five six seven eight nine ten" are 10 words also.

Indeed. But I must wonder why there are so many decent reviews on Amazon for example (decent as in, no one-word reviews). Or why lots of posts in this forum actually contain information that is worthy of being read. Is it the layout? Or the site's reputation? Target audience? Good moderators (of course)?
Ony tiny step into the right direction could be to use more complex mechanics or choice of words. Not "vote" but "review" or something along those lines..
_________________
“Through the sounds of falling rain, through the clouds of bitter times
I see the pure grace of your smile, in dreams of the warmth in your eyes”
- Tim North
Post Sun Jan 18, 2004 9:21 pm
 View user's profile
Myrthos
Spoiler of All Fun
Spoiler of All Fun




Joined: 07 Jul 2001
Posts: 1926
Location: Holland
   

[quote="Ariel"]Indeed. But I must wonder why there are so many decent reviews on Amazon for example (decent as in, no one-word reviews). Or why lots of posts in this forum actually contain information that is worthy of being read. Is it the layout? Or the site's reputation? Target audience? Good moderators (of course)?[quote]
I think those reviews are linked to the same account you use for purchasing stuff on Amazon. Or in other words, Amazon has your address and other account info when you put that review up. I think that would inhibit some to put up crap.
_________________
Kewl quotes:
I often have an odd sense of humor - Roach
Why quote somebody else, think of something yourself. - XeroX
...you won't have to unbookmark this site, we'll unbookmark you. - Val
Reports Myrthos for making me scared and humbled at the mere sight of his name - kayla
Post Sun Jan 18, 2004 10:42 pm
 View user's profile
goshuto
Wanderer
Wanderer




Joined: 29 May 2002
Posts: 1142
   

I also think Amazon has people to read each review before it's actually allowed to be posted. That's why it takes time for reviews to appear on the site. Heck, if nobody else is willing to do this, I will. I can "review the reviews." But what guidelines should be used to consider a review "crap?"
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard
Post Mon Jan 19, 2004 2:54 pm
 View user's profile
Lintra
Elf Friend
Elf Friend




Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES
   

quote:
Originally posted by goshuto
I also think Amazon has people to read each review before it's actually allowed to be posted. That's why it takes time for reviews to appear on the site. Heck, if nobody else is willing to do this, I will. I can "review the reviews." But what guidelines should be used to consider a review "crap?"


All hail goshuto, the selfless sacrificer of his time for the greater good!

In a fit of guilt at not having done much for RPGDot recently, I will volunteer to help goshuto with this task ... if it is possible.

@goshuto - I have heard it said that misery loves company, so I will try to help you if you get stuck with this honor. I am not all that reliable time-wise since my job can have me AWOL from an internet connection for weeks at a time, but I will help as best I can.
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless=
Post Mon Jan 19, 2004 3:00 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

quote:
Originally posted by dteowner
It will have to be several months from now, but I'll volunteer to tackle that project. My free time right now is, and must continue to be, dedicated to my job search. Once that is completed, I'll jump on that tiger.
*cough*

Looks like we have a team building!

I still think we need to generate some guildelines so people have some idea what a score means. For instance, I'd say that each person should only give 3-4 games a 10 (trusting people to behave), since a 10 should only go to "one of my favorite games of all time". Very, very good games that don't make the "all time" cut should be given a 9. And so forth. In my mind, the "average good game" should be getting a 7. Scores below 5 should be saved for flawed and badly bugged games. A 1 should only be given to a game that is so buggy that it's completely unplayable, which applies to something like 5 games ever.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Mon Jan 19, 2004 4:21 pm
 View user's profile
goshuto
Wanderer
Wanderer




Joined: 29 May 2002
Posts: 1142
   

quote:
Originally posted by dteowner
*cough*


It wasn't my intent to ignore you, dte. My apologies if it sounded like that. I just volunteered to start working away.

quote:
Originally posted by Lintra
All hail goshuto, the selfless sacrificer of his time for the greater good!


Are you making fun of me? Seriously speaking, thanks for the offer of help. If we're building a team, as dte pointed out, this job can be done in no time! (and I still volunteer to "review future reviews")
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard
Post Mon Jan 19, 2004 4:52 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

It's, OK, Goshuto. I was just teasing.

Myrthos, it sounds to me like we've got a team coming together here. As I mentioned, I won't be able to devote the proper amount of time to this for a while, but we can certainly do the planning phase immediately. Shall we have an official sanction?
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Mon Jan 19, 2004 5:45 pm
 View user's profile
Gorath
Mostly Harmless
Mostly Harmless




Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany
   

I doubt it´s possible to review more than 10k reviews by hand. That´s why I think we should work on the system. A few randim thoughts:
- If you look at the reviews on most sites it´s clear that a 50% or 5/10 score is an exception nowadays. Only complete crap gets such a score. Therefore I think we should start at 5 and go up in .5 steps. This results in higher scores and leads to a misrepresentation of utterly crappy games, but it significantly decreases the impact of fanboy reviewers who think they need to give a popular game a 1/10 because 'it´s not NWN'. This would be a serious improvement IMHO.
- An explanation should be added to the score. For example '7 - Average. I got my money´s worth, but not more.'
- An info text should be added to make clear what´s the list is about and that manipulations will be removed.
- comment:'comment' and the like should be filtered out.
- Maybe we should set a minimum wordcount to 10 or 20.
- All scores from 1-3 for all games should be checked.
_________________
Webmaster GothicDot
Post Mon Jan 19, 2004 5:55 pm
 View user's profile
goshuto
Wanderer
Wanderer




Joined: 29 May 2002
Posts: 1142
   

The problem with letting anyone post anything is that there are some reviews which contain "questionable" words and expressions, if you catch my drift. I don't think it would be in RPGDot's best interest to leave them there. If it's not possible to check all past reviews, as Gorath suggests, then at least future reviews should get "approval" before they're posted on the site.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard
Post Mon Jan 19, 2004 6:11 pm
 View user's profile
Lintra
Elf Friend
Elf Friend




Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES
   

quote:
Originally posted by goshuto
quote:
Originally posted by dteowner
*cough*


It wasn't my intent to ignore you, dte. My apologies if it sounded like that. I just volunteered to start working away.

quote:
Originally posted by Lintra
All hail goshuto, the selfless sacrificer of his time for the greater good!


Are you making fun of me? Seriously speaking, thanks for the offer of help. If we're building a team, as dte pointed out, this job can be done in no time! (and I still volunteer to "review future reviews")


Umm, errr, *cough*, I was only volunteering to help do future reviews! Reading all the reviews currently out there is .. well ... not really doable. A sub set created by some sort of screening script I would be willing to volunteer to help with though.
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless=
Post Mon Jan 19, 2004 6:13 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

Personally, I'd favor flushing the old data and starting over. People that are actually trying to do a good job with their reviews might want to change their numerical scores based on any new system we put in place.

A side benefit of starting over is that we could stagger the new entries. We could allow a handful of people to put in their reviews prior to opening up the floodgates to the general membership. This would help the "review police", as well as giving the new system a proper test run before full implementation.

The downside, obviously, is that we'd be tossing a lot of info.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Mon Jan 19, 2004 6:18 pm
 View user's profile
Val
Risen From Ashes
Risen From Ashes




Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA
   

quote:
Originally posted by Gorath
I doubt it´s possible to review more than 10k reviews by hand.

Actually, it is. I've done it once already. I looked for suspicious scores and behavior from people who reviewed multiple times. I even found some instances where the score given didn't match the review and we were able to e-mail the person and verify if that was the score they actually meant to give.

We don't have to dump the current data. Divide it up into smaller chunks and have our "team" go through them. Throw out all scores that don't match our new system, like scores with no comments, etc.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=
Post Mon Jan 19, 2004 6:52 pm
 View user's profile
dteowner
Shoegazer
Shoegazer




Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia
   

How many games have you given 10's to? Let's say the answer is 6. By the "new system" (which doesn't even exist yet, but humor me), some of those need to be kicked down to 9's. The "team" (which doesn't officially exist yet, but humor me ) would have no way of defining which 3 you wanted to be 10's and which should be demoted. I'm not sure how practical it would be to ask everyone to revisit their choices. Of greater concern, how many people have given any game they liked enough to finish an 8 or 9? The "new system" would dictate many of those should be getting in the 6-8 range.

Going to new, consistent scoring criteria makes it extremely difficult to judge the integrity of older data.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys!
Post Mon Jan 19, 2004 7:34 pm
 View user's profile
Val
Risen From Ashes
Risen From Ashes




Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA
   

It could also anger our users if we dump all of their reviews. If you took the time to write a good review for several games, then how would you like it if someone just flushed it all down the toilet just because they didn't like the results?
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=
Post Mon Jan 19, 2004 7:55 pm
 View user's profile
Lintra
Elf Friend
Elf Friend




Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES
   

@Val - I don't think anyone (outside of dte) is seriously considering dumping all the previous reviews.

@all - how hard would it be to count the number of reviews with no commentary, 5 words or less, 10 words or less, and 20 words or less?

I would be interested to know. If it turns out that there are 1,000 reviews w/ no words, then those should be killed. If it further turns out that there are 762 reviews w/ 10 or fewer words, I (for one) would volunteer to help go through them and kill the obvious dumb ones.

Then there would be the committee to review all future reviews (I have already volunteered for this) ... I would picture it working as follows: after a review is submitted it joins a queue. Then some good samaritan logs on, and goes through the queue rejecting or accepting each review. After being accepted, it becomes part of the data base and is included in all future stats from that time forward.

@Dte - I don't thing there needs to be hard and fast rules on the reviews ... upon reading it should be self evident whether or not the submitter is gaming the system ... or being a fan boy.
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless=
Post Mon Jan 19, 2004 8:03 pm
 View user's profile


Goto page Previous  1, 2, 3, 4, 5, 6, 7, 8, 9  Next
All times are GMT.
The time now is Thu Apr 11, 2019 9:51 am



Powered by phpBB © 2001 phpBB Group
 
 
 
All original content of this site is copyrighted by RPGWatch. Copying or reproducing of any part of this site is strictly prohibited. Taking anything from this site without authorisation will be considered stealing and we'll be forced to visit you and jump on your legs until you give it back.