| |
Site Navigation
Main
News
Forums
Games
Games Database
Top 100
Release List
Support Files
Features
Reviews
Previews
Interviews
Editorials
Diaries
Misc
Download
Gallery
Music
Screenshots
Videos
Miscellaneous
Staff Members
Privacy Statement
|
|
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
quote:
Originally posted by Val
It could also anger our users if we dump all of their reviews. If you took the time to write a good review for several games, then how would you like it if someone just flushed it all down the toilet just because they didn't like the results?
You didn't answer my earlier question, which is rather critical to any further discussion on the issue. You also are saying I don't like the results, when I couldn't give a hoot about the results themselves. I'm just offering to act on what our users have determined to be a problem. The current system does not serve its purpose (as evidenced by this thread), so I'm offering to help fix it. If you have a better, effective idea, the non-existant committee is hiring.
To answer your question, I've done a great deal of work on things that gets thrown in the trash because the rules change after the fact. It's nothing personal, and I've somehow managed to keep my tears to a minimum. For that matter, the conscientious people that have kindly taken their time to do quality reviews would probably also be the people that want the system to have value, where a score actually has definite meaning. That serves other users, which is the purpose of reviews in the first place, yes?
@Lintra- your queue idea is exactly what I've envisioned as well. As long as it's do-able, I think the non-existant committee has managed its first non-existant recommendation.  Also, I have no desire to dictate rules on the text reviews. I don't really have trouble with "reviews" that have a numerical score and no text, as long as the score actually means something. In my mind, where a score in the 6-8 range is typical of a "good" game, only scores outside that typical range require explanation. The further one strays from the 6-8 range (good or bad), the more explanation I'd expect to see. A "10" with nothing to describe why that game is mana from heaven would/should be tossed just as quickly as a "1". Likewise, a review of a "Fun, but nothing special" seems sufficient for a game that is given a 7.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
Mon Jan 19, 2004 10:52 pm |
|
|
Lintra
Elf Friend
Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES |
I realized I'd not replied to Gorath before (sorry about that Gorath, I thought I had, but those paragraphs got lost somehow 
quote:
Originally posted by Gorath
I doubt itīs possible to review more than 10k reviews by hand. Thatīs why I think we should work on the system. A few randim thoughts:
- If you look at the reviews on most sites itīs clear that a 50% or 5/10 score is an exception nowadays. Only complete crap gets such a score. Therefore I think we should start at 5 and go up in .5 steps. This results in higher scores and leads to a misrepresentation of utterly crappy games, but it significantly decreases the impact of fanboy reviewers who think they need to give a popular game a 1/10 because 'itīs not NWN'. This would be a serious improvement IMHO.
Hmm, not sure about this point. I don't really like scoring 'on a curve' just because it is expected .. let the law of large numbers do it for us.
quote:
- An explanation should be added to the score. For example '7 - Average. I got my moneyīs worth, but not more.'
yes. Absoloutely
quote:
- An info text should be added to make clear whatīs the list is about and that manipulations will be removed.
Absolutley.
quote:
- comment:'comment' and the like should be filtered out.
Goes with out saying *grin*
quote:
- All scores from 1-3 for all games should be checked.
Dunno about this ... it sounds like a good idea, but how much work is it? Also, what about artificially inflated scores?
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless= |
|
Mon Jan 19, 2004 11:31 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
quote:
Originally posted by dteowner
You didn't answer my earlier question,
Fine. If you want to get hit over the head with a frying pan, then who am I to stop you?
quote:
Originally posted by dteowner
How many games have you given 10's to?
A few. Not six. Since I participated fairly, I see no reason why I should be punished for someone else's poor behavior.
quote:
Originally posted by dteowner
You also are saying I don't like the results, when I couldn't give a hoot about the results themselves.
I did not make this personal. But you just did.
I was looking at things from a possible point of view of our visitors. They probably won't feel all that welcome when we do the equivilent of not even bothering to pretend that their opinions matter to us. By sorting through the current database of reviews it is possible to pick out worthwhile reviews of the games. There are many of them and they shouldn't just be dropped by the wayside because it is convenient.
quote:
Originally posted by dteowner
I'm just offering to act on what our users have determined to be a problem.
No. You are overreacting to something that one user pointed out and a handful of others have agreed with.
quote:
Originally posted by dteowner
If you have a better, effective idea, the non-existant committee is hiring.
I do have a better idea which involves more work. You just don't seem to like it.
quote:
Originally posted by dteowner
For that matter, the conscientious people that have kindly taken their time to do quality reviews would probably also be the people that want the system to have value, where a score actually has definite meaning.
Again, it is unjust to punish those who participated fairly. And some could very much feel that way about it. We'd do them a disservice by not even trying to initially filter the current reviews.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=  |
|
Tue Jan 20, 2004 5:09 am |
|
|
corwin
On the Razorblade of Life
Joined: 10 Jun 2002
Posts: 8376
Location: Australia |
I have a question. How many people actually read these reviews? I visited the 'Top 100' for the first time today to see what all the fuss was about. I looked at one game in depth and saw very few 'reviews' that were over one sentence long. Some scores and comments were obviously written by people with a mental age of 2, but aside from a quick look to see what games are popular, how many people would really spend the time to look at all the comments? I was surprised at how low games like Gothic were rated, but most of the top 20, at least, seemed reasonable. Why not have a quick poll to determine how many would be in favour of a clean slate? Finally, some wise words you've all heard before; try to be part of the solution, not the problem!!
_________________
If God said it, then that settles it!
I don't use Smileys, I use Emoticons!!
 |
|
Tue Jan 20, 2004 8:04 am |
|
|
aldreth
Eager Tradesman
Joined: 09 Feb 2003
Posts: 26
Location: Washington |
I just wanted to let you all know that this response is the exact reason I have been coming to RPGDot.com for years. The RPGDot team has always astounded me by listening and giving invaluable help to their viewers. Ten thumbs up! =)
I just wanted to add an idea that I stated a bit earlier. Would it be possible to make people register at RPGDot before they voted and added their review? To me, I think that would definitely cut down on the "dishonest" reviews that inevitably make the list.
_________________
`''''''
(0 0)
-(_)--oOO---------
Work to become,
not to acquire.
-----------oOO-----
__|__|
|| ||
oO oO |
|
Tue Jan 20, 2004 9:01 am |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
quote:
Originally posted by aldreth
Would it be possible to make people register at RPGDot before they voted and added their review? To me, I think that would definitely cut down on the "dishonest" reviews that inevitably make the list.
It would certainly cut down those dishonest reviews, but it would also significantly cut down the number of reviews. At this point, I'm not sure that's a good or bad thing.
Deleting all current reviews is not really an option, in my opinion. Those 10K reviews are an asset to this site. The fact that people even bothered to write something means they like this site above others. Many site owners would give an arm and a leg to generate this kind of interest for their own site. When visiting a site, most users simply skim over the contents and move on to other sites. When they actually take their time to post something on a site, it creates some sort of link: their opinion is now on display there, and they will certainly come back to that site later to check for possible feedback. People returning to websites is a good thing. Hence, the reviews must stay, in my opinion. If that means going over all 10K of them, so be it. As I said, I volunteer for it. And a system which requires them to be reviewed before they actually appear on the site should be implemented, as that would benefit RPGDot greatly.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Tue Jan 20, 2004 3:19 pm |
|
|
Gorath
Mostly Harmless
Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany |
Agreed, 10k reviews are an asset. Deleting them is no option.
The pipeline thing is a good idea. If properly communicated it will discourage manipulated reviews because the author has to assume it wonīt make the cut.
_________________
Webmaster GothicDot |
|
Tue Jan 20, 2004 6:24 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
quote:
Originally posted by Val
I did not make this personal.
Bullcrap. Let me refresh your memory:
quote:
Originally posted by Val
It could also anger our users if we dump all of their reviews. If you took the time to write a good review for several games, then how would you like it if someone just flushed it all down the toilet just because they didn't like the results?
Just who are you talking to/about, if it ain't me? After all, it was my idea. So let's just cut the innocent routine. It's tiresome, counterproductive, and blatantly false.
quote:
Originally posted by Val
I do have a better idea which involves more work. You just don't seem to like it.
I'm still waiting to hear it. So far, as I've told you directly, you've only put up problems, but no solutions. Merely going thru 10k reviews doesn't address the core, systemic problem with the numerical scores which I have raised and attempted to remedy.
If you want the reins, grab hold. Last time I checked, here was your application:
quote:
Originally posted by Val
*Val remembers looking through all of the votes once to pick out the bad stuff. Val hides.*
I'll help or get the heck out of your way if you want to tackle it. If you don't want to deal with the ENTIRE problem, then please stop taking potshots at me so I can help the non-existant committee get something done.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Tue Jan 20, 2004 6:41 pm |
|
|
Gorath
Mostly Harmless
Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany |
quote:
Originally posted by Lintra
I realized I'd not replied to Gorath before (sorry about that Gorath, I thought I had, but those paragraphs got lost somehow
Yeah, the old alternate reality problem.
quote:
Hmm, not sure about this point. I don't really like scoring 'on a curve' just because it is expected .. let the law of large numbers do it for us.
Okay, let me try it the other way round. RPGDot is a site with very high review score (of course partly due to political reasons). The lowest I could find was a 64% for Elixir. The failed M&M 9 got 82%. wouldnīt it be in an improvement to make our reader reviews comparable to the editor reviews?
I still think nowadays everything below 70% is perceived as low. If so we have 6 rating options representing low, 1 for average and 3 for good to great. Sounds improvable to me.
quote:
- All scores from 1-3 for all games should be checked.
quote:
Dunno about this ... it sounds like a good idea, but how much work is it? Also, what about artificially inflated scores?
Artificially inflated scores are more problematic. A 'best game ever!' is correct for 10 points.
_________________
Webmaster GothicDot |
|
Tue Jan 20, 2004 6:43 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
One other thing, real quick. If slogging thru 10k reviews to weed out the garbage is a good solution, why are we having to do it repeatedly? Val spent a huge amount of effort cleaning out the crap a while back and here we are again. Wouldn't it make more sense to come up with a different solution that keeps the garbage from ever getting into the system in the first place?
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Tue Jan 20, 2004 6:56 pm |
|
|
Gorath
Mostly Harmless
Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany |
Absolutely. Thatīs why weīre still talking instead of start cleaning.
_________________
Webmaster GothicDot |
|
Tue Jan 20, 2004 7:01 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
quote:
Originally posted by dteowner
Wouldn't it make more sense to come up with a different solution that keeps the garbage from ever getting into the system in the first place?
The "pipeline" system does exactly that: no review ever gets on the site without approval.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Tue Jan 20, 2004 7:07 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
It's not a routine, dteowner. I was giving you an example of the possible negative feelings that the action of deleting all of the reviews could create. It's wasteful to delete it.
quote:
Originally posted by dteowner
I'm still waiting to hear it.
Fine. I'll spell it out for you.
Step 1: Impliment new rules for the Top 100 list. i.e. Word count, new grading scale, etc.
Step 2: Put together the team.
Step 3: Create the "pipeline" so new reviews can be filtered.
Step 4: Divide up the current reviews amoung the team members and have them go over them to pick out the ones that will fit in the new system. Dividing the workload will allow us to filter through the current reviews much quicker.
That's my idea. It can work.
quote:
Originally posted by dteowner
If you want the reins, grab hold. Last time I checked, here was your application:
quote:
Originally posted by Val
*Val remembers looking through all of the votes once to pick out the bad stuff. Val hides.*
Cute, lost your sense of humor right along with your sense of tact, I see? 
I'm more than happy to help if you'll stop attacking me.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Tue Jan 20, 2004 7:23 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
quote:
Originally posted by Val
Fine. I'll spell it out for you.
Step 1: Impliment new rules for the Top 100 list. i.e. Word count, new grading scale, etc.
Step 2: Put together the team.
Step 3: Create the "pipeline" so new reviews can be filtered.
Step 4: Divide up the current reviews amoung the team members and have them go over them to pick out the ones that will fit in the new system. Dividing the workload will allow us to filter through the current reviews much quicker.
That's my idea. It can work.
Looks a great deal like what the non-existant committee already had brainstormed before you started telling me I needed hit with a frying pan. Now that you've finally put a solution up, let's work with it. That's all I've asked of you from the very beginning. It's unfortunate you felt the need to be nasty in getting there.
Now, it seems to me that we've been addressing some of the actual details of your step 1 while you've been yelling from the sidelines. If we institute a new scoring system, how will you impose it on the existing data? If you don't correct the old data, then the system is hugely flawed before it ever gets started. On the other hand, there's some real problems with fixing it. You're concerned about upsetting people by starting over (which is certainly a valid concern), but you're going to allow the committee to go in and alter someone's numerical score to make it conform to the new rules? I would think you'd be concerned about the impact on our users for altering or arbitrarily deleting their reviews. Or were you in favor of maintaining the existing info intact, knowing that doesn't fix anything?
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Tue Jan 20, 2004 7:47 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
Since you seem determined to make this harder than it is, I'll elaborate.
The scoring system proposed is to simply add half points. Obviously:
code:
Old system scores New system scores
10 = 10
9 = 9.5
8 = 9
7 = 8.5
6 = 8
5 = 7.5
4 = 7
3 = 6.5
2 = 6
1 = 5.5
Just like Ariel pointed out earlier. This makes it so the low scores do not weigh so heavily on the overall score. That way the fanboys aren't as motivated to drop 10s on all of their favorite games just to try and overcome a 1 which was unfairly given.
All you'd have to do is look at the score the reviewer gave and then convert it over. Simple. You keep the old data worth having and integrate it into the new system instead of just dumping it as you proposed. See?
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound= |
|
Tue Jan 20, 2004 8:04 pm |
|
|
|
Goto page Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next
All times are GMT. The time now is Thu Apr 11, 2019 9:51 am
|
|
|
|
|
|
|
