| |
Site Navigation
Main
News
Forums
Games
Games Database
Top 100
Release List
Support Files
Features
Reviews
Previews
Interviews
Editorials
Diaries
Misc
Download
Gallery
Music
Screenshots
Videos
Miscellaneous
Staff Members
Privacy Statement
|
|
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
So we're going to "grade on the curve" without adding any real meaning to the scale itself? Why bother? If we're going to undertake a major effort, wouldn't it make sense to do it right so you're not resorting to frying pans again in 6 months when we have to go in and edit 15,000 reviews with numerical scores that mean next to nothing?
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
Tue Jan 20, 2004 8:39 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
How much more obvious must I make this? 
We can go ahead and elaborate on the meaning of each number in the scoring system as previously suggested. So, it'll appear to the outside user to be a normal 1 to 10 scale. For example:
10 - Greatest game ever.
9 - Definately worth the money and more.
8 - Good overall game.
7 - One to check out for those that like the genre.
6 - Worth the price of admission if you like the genre.
5 - Average, wait for it to hit a bargin bin near you.
4 - Flawed by bugs, wait for the patches.
3 - Not recommended for those without masochistic tendencies.
2 - My eyes! They burn!!!
1 - Do not under any circumstances buy this game! Not even if they're holding a gun to your head.
But when the review is accepted from the pipeline and added to the existing pool the overall score is calculated on the curve. Thus individual "bad" scores won't drag down the overall grade of the game, if that bad score is accepted at all. This would mostly affect the highly popular games that are voted on the most like Gothic or Fallout, etc. Those would be the ones that have been the most effected.
You can add exceptions to the curve scoring to account for actual bad games.
And that's if it's even decided to implement a new scoring system.
We could just go with the word limit, reviewing the current reviews and the pipeline. These are all possible solutions. My main point of contention, if you'll remember, is that it isn't nessecary to drop all of the reviews that we already have. Current reviews can be found in the database that could fit the criteria once it is decided upon.
With the pipeline, the reviews will be "moderated" before they actually get submitted. Thus, the problem won't be happening again 6 months down the road as goshuto already pointed out.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=  |
|
Tue Jan 20, 2004 9:20 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
quote:
Originally posted by Val
How much more obvious must I make this?
Well, it only took me 5 posts or so to get you to actually spit the whole thing out. Guess you finally hit the finish line that the non-existant committee crossed 30 posts ago while you were busy looking for that frying pan.
Note that you still haven't determined how to make 10000 existing numerical scores conform to the scale you just defined (which I think is excellent, by the way, and matches up rather well to the one I already proposed). It's highly impractical to email each reviewer to ask them to make certain their old scores match up with the new scale, and I don't think you are in favor of the committee autocratically doing it for them. Remember when I asked how many 10's you'd given? That was a "generic you", although it doesn't really matter- it might apply to Val; it most certainly applies to some reviewers, possibly even some highly conscientious ones. That's the issue that got this whole steaming pile of wasted electrons going in the first place!!! To paraphrase, how much clearer must I state this?
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Tue Jan 20, 2004 9:57 pm |
|
|
Lord Chambers
Eager Tradesman
Joined: 22 Dec 2003
Posts: 29
|
I'd just like to express my love of these forums and the seeming intelligence of the average poster. I've never seen so much quality discussion filled with so much correct punctuation, capitalization, and grammar.
And since anything I had to say about the voting system has already been said, I'll just let you guys carry on.  |
|
Tue Jan 20, 2004 11:26 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
*sigh*
After determining the criteria by which the new Top 100 will exist, then the "team" can use that criteria to sort through the current reviews. As I already stated. It's a little hard to give you an exact methodology until the actual criteria is determined. But if you'd like an example for the sake of arguement, here's one.
Phase #1: Initial sorting.
Criteria #1: Reviews must contain no less than 10 words.
Criteria #2: The review must be an actual review. Not just a string of words to fullfill the word limit or rant of swear words expressing that particular user's displeasure with the game. An actual review should be an explaination of the score given.
By those criteria alone one can begin to sort through the current reviews. Reviews that contain nothing but "comment" or other nonsense would automatically be eliminated from the list of potential revews to be converted to the new system. The reviews that show that the user took some time, effort and thought when making their comments are preserved.
This would eliminate from the system the sort of votes that the initial poster was concerned with.
Phase #2: Implimentation of possible new scoring system
Criteria #1: See previous posts concerning "grading curve".
After the initial filtering this scoring on a curve would probably only have to be applied to the top twenty or so games. Since those games are subject to the most attempts at manipulation in the Top 100 due to the "fanboy factor".
Also, the fact that a game needs at least 10 reviews to appear on the Top 100 prevents any one user from simply voting some obscure game to the top by giving it a 10. that's a good rule that should be kept for the new system.
While the filtering is taking place, access to the Top 100 should be removed so those going over the current set of reviews don't have to deal with new ones popping up. Once this filtering of the current reviews is done and the refuse has been discarded, then the "pipeline" can be added and access to the Top 100 can be restored. As stated before, reviews won't be automatically added in. They'll go to the cue and await approval by the team. New reviews will be judged by the new criteria and whatever rankings are applied the number system I stated in my previous post.
Simple.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Wed Jan 21, 2004 12:28 am |
|
|
corwin
On the Razorblade of Life
Joined: 10 Jun 2002
Posts: 8376
Location: Australia |
Not simple Val; that's been the problem. Your ideas have much merit and I expect most if not all will be implemented. If your last couple of posts had been your first couple, this thread would probably be two pages shorter!! 
_________________
If God said it, then that settles it!
I don't use Smileys, I use Emoticons!!
 |
|
Wed Jan 21, 2004 1:47 am |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
I like that system. I will have to partly disagree with you corwin, and say it is a simple system, but it gets the job done; more complex systems could be hard to implement (poor Myrthos  ), and would not be necessarily better. Further, it requires little messing around with the current grades.
Speaking of simple, I think the scoring scale could be simplified. Too many options tend to confuse users. Confused users is not something RPGDot would want. And yes, sometimes ten options can be considered too much. I propose reducing the number of grades each user can choose from.
"That would reduce the granularity, goshuto!"
Yes, it would. But let's ask ourselves: is all that granularity really necessary? For instance, does it really matter if a user rates a game a 2, 3, or 4? All three grades pretty much mean the user hated the game, in my opinion. So why not reduce all of them to a single category, say, "Not recommended for those without masochistic tendencies." (because that sounds cool ), and assign, say, 3 to that category? We could further merge 6,7 and 8, for example. Using Val's scale and working from there:
10 - Greatest game ever.
9 - Definately worth the money and more.
6,7,8 - Good overall game.
5 - Average, wait for it to hit a bargin bin near you.
2,3,4 - Not recommended for those without masochistic tendencies.
1 - Do not under any circumstances buy this game! Not even if they're holding a gun to your head.
The result is that we get to keep the current numerical scale, we get less confused users (and as a result, possibly slightly more accurate ratings), and the downside is that a game never gets rated a 2, or 4, or 6. But is this really a problem? I don't think so. A 3 will do enough to get the rating down, and a 7 will do enough to increase it.
And I would even go as far as suggest removing the numbers from the scale that is shown to the user, as there is a chance that the text will be completely ignored -- single digit numbers are so much easier to read. And a rating of 8 does not necessarily mean the same thing for all people.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 2:22 am |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
@corwin: Stop trying to sow the seeds of contention. It is not helping.
@goshuto: I find reducing the granularity appealing as well. Sometimes there is such a thing as too much of a good thing.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Wed Jan 21, 2004 3:53 am |
|
|
Gorath
Mostly Harmless
Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany |
Since I suggested simplifying the rating system some pages ago I´m of course all for it.
Combining the low scores is no problem, but how do you do it with higher scores? Sometimes there´s a reason why a reviewer gave a game a 7 instead of an 8. This would automatically lead to misrepresentations. I´m not sure if this would be respectful towards our readers.
_________________
Webmaster GothicDot |
|
Wed Jan 21, 2004 7:58 am |
|
|
corwin
On the Razorblade of Life
Joined: 10 Jun 2002
Posts: 8376
Location: Australia |
@goshuto I meant finding a workable solution wasn't simple, not that a complex one was needed.
@Val I wasn't being contentious, that's not my style. I was merely making an astute observation. If the boot fits, learn to wear it!! 
_________________
If God said it, then that settles it!
I don't use Smileys, I use Emoticons!!
|
|
Wed Jan 21, 2004 8:16 am |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
If that's the case, then you and dteowner should put your boots on.
@Gorath: I agree that would be a problem. The lowest 4 to 5 scores are simple to combine because usually they all mean the same thing. The player didn't like the game. At all. And they find it very difficult to recommend the game. Perhaps if we could look at a cross-section of scores from the Top 100?
Myrthos, is it possible to pull some statistical data from the Top 100? Like how many 1s were given and how many 6s, 7s, 8s, etc were given?
That could help us determine whether it is plausible to easily convert the grades readers have given over to a simplier system.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Wed Jan 21, 2004 12:01 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
quote:
Originally posted by Gorath
Combining the low scores is no problem, but how do you do it with higher scores?
Note that, with the system I proposed, there are more "high scores" than "low scores", precisely because of the reason Val mentioned: a low grade is a low grade, and there is little point, in my opinion, in differentiating a 2 from a 3 or 4. Besides, as you mentioned yourself, scores on the web tend to be inflated.
quote:
Originally posted by Gorath
Sometimes there´s a reason why a reviewer gave a game a 7 instead of an 8. This would automatically lead to misrepresentations. I´m not sure if this would be respectful towards our readers.
It would not necessarily lead to misrepresentations. I'm willing to bet that few readers apply a specific methodology when rating a game. That is, I believe most don't try to separately rating graphics, sound, gameplay, etc. and then derive the score from there (or apply some other similar method). I'm of the opinion that most simply guess a score: "oh, it was a good game. I guess 8 sounds about right." This is what I believe causes misrepresentation. That's why I think numerical ratings should not be shown to the users at all. I'm sure there are reviewers who apply "correct" methodology to rate a game, but as I said, I'm willing to bet most don't bother with it at all.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 12:13 pm |
|
|
Myrthos
Spoiler of All Fun
Joined: 07 Jul 2001
Posts: 1926
Location: Holland |
From a total of 10,190 votes:
1. 866
2. 225
3. 225
4. 222
5. 354
6. 412
7. 610
8. 913
9. 1,466
10. 4,897
_________________
Kewl quotes:
I often have an odd sense of humor - Roach
Why quote somebody else, think of something yourself. - XeroX
...you won't have to unbookmark this site, we'll unbookmark you. - Val
Reports Myrthos for making me scared and humbled at the mere sight of his name - kayla |
|
Wed Jan 21, 2004 12:13 pm |
|
|
corwin
On the Razorblade of Life
Joined: 10 Jun 2002
Posts: 8376
Location: Australia |
goshuto, just wanted to make a quick comment on your new sig. Cute, but we all know the problem with dividing by zero, dont we!!
Val, I refuse to dignify your retort with any reply whatsoever!!
_________________
If God said it, then that settles it!
I don't use Smileys, I use Emoticons!!
Last edited by corwin on Wed Jan 21, 2004 12:21 pm; edited 1 time in total |
|
Wed Jan 21, 2004 12:19 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
@corwin: Then why did you just reply?
Thanks Myrthos!
Very interesting. Well, it seems to support the theory that 2, 3, and 4 seem to be ignored in favor of 1 by a majority of users.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Wed Jan 21, 2004 12:19 pm |
|
|
|
Goto page Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next
All times are GMT. The time now is Thu Apr 11, 2019 9:51 am
|
|
|
|
|
|
|
