| |
Site Navigation
Main
News
Forums
Games
Games Database
Top 100
Release List
Support Files
Features
Reviews
Previews
Interviews
Editorials
Diaries
Misc
Download
Gallery
Music
Screenshots
Videos
Miscellaneous
Staff Members
Privacy Statement
|
|
|
|
corwin
On the Razorblade of Life
Joined: 10 Jun 2002
Posts: 8376
Location: Australia |
I do NOT consider that a reply!!
_________________
If God said it, then that settles it!
I don't use Smileys, I use Emoticons!!
 |
Wed Jan 21, 2004 12:29 pm |
|
|
Val
Risen From Ashes
Joined: 18 Feb 2002
Posts: 14724
Location: Utah, USA |
Uh huh. Whatever.
Thinking about it, the statistical data does prove Gorath's point that the 7s and 8s are valuable scores to keep seperated. While the bottom half of the scoring isn't well utilized, the upper half appears to be. And that makes shortening it a lot harder.
EDIT: Then again, it does lend itself to the grading scale.
A = 9 or 10
B = 8
C = 7
D = 6
F = 5 or lower
Sort of works. But I'm not entirely happy about 9 and 10 both being an A. there needs to be a distinction like an A+ for a 10 or something.
_________________
Freeeeeeedom! Thank heavens it's summer!
What do I have to show for my hard work? A piece of paper! Wee!
=Guardian, Moderator, UltimaDot Newshound=  |
|
Wed Jan 21, 2004 12:39 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
That data suggests that ratings from 1 to 8 only make up ~37% of all scores. A rating of 5, which supposedly means an "average" game (taking into consideration a scale of 1 to 10), only accounts for 3.5% of all ratings. A rating of 10 accounts for ~48%. This means that (ignoring dishonest ratings), if we classify a rating of 10 as "best game ever made," roughly half of the users would have voted because they really, really liked a game. Which probably makes sense, anyway. Since 6, 7, and 8 together account for 19%, they could perhaps be split into two ratings? But that leaves 5 (3.5%) all by itself, not necessarily a good thing. Then:
1 -worst
2,3,4
5 (?)
6,7 (6.5?)
7,8 (7.5?)
9
10 -best
Or maybe just ignore this and clump all low scores together (which could, conceivably, lead to an even larger number of "good votes" and skew the grades even more towards 10).
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 1:00 pm |
|
|
Myrthos
Spoiler of All Fun
Joined: 07 Jul 2001
Posts: 1926
Location: Holland |
We can also do it in stars, ranging from a half to 5 stars, with steps of a half star.
10 = 5 stars
9.5 = 4.5 stars
9 = 4 stars
8.5 = 3.5 stars
8 = 3 stars
7.5 = 2.5 stars
7 = 2 stars
6.5 = 1.5 stars
6 = 1 star
Everything below that is 0.5 star
Most games should be above a 6 anyway to be even remotely considered to be worth playing, so adding a few steps could make things easier.
_________________
Kewl quotes:
I often have an odd sense of humor - Roach
Why quote somebody else, think of something yourself. - XeroX
...you won't have to unbookmark this site, we'll unbookmark you. - Val
Reports Myrthos for making me scared and humbled at the mere sight of his name - kayla |
|
Wed Jan 21, 2004 1:09 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
That is a good idea, but what grade do you assign to 0.5 star? If you assign 5 to it, that means even the most awful of games will get a rating of at least 50%. If you assign 1 to it, the problem of fanboys artificially lowering a score of some games also returns. Maybe assign a 3 to it?
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 1:16 pm |
|
|
Myrthos
Spoiler of All Fun
Joined: 07 Jul 2001
Posts: 1926
Location: Holland |
Well you don't give your rating in mumbers anymore, but in stars.
_________________
Kewl quotes:
I often have an odd sense of humor - Roach
Why quote somebody else, think of something yourself. - XeroX
...you won't have to unbookmark this site, we'll unbookmark you. - Val
Reports Myrthos for making me scared and humbled at the mere sight of his name - kayla |
|
Wed Jan 21, 2004 1:43 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
Ah, I see now. But do the ratings of the Top 100 also get converted to stars? If so, that could cause problems (look at how the top games are in the 8.5-9 range). If you don't convert the ratings themselves to stars, then you will still have to convert the user ratings stars to numbers.
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 1:53 pm |
|
|
mkreku
Keeper of the Gates
Joined: 22 Oct 2003
Posts: 112
Location: Uppsala, Sweden |
I looked through the "reviews" for the game Gothic 2 on the Top 100-list and I noticed that almost everyone who gave the rating 1/10 to Gothic 2 has only ONE registered review in the database, meaning they're obvious attempts at bringing the game down in the rankings (and it looks like they've succeeded since Gothic 2 ranks 40+ now)..
What if you made the database NOT count the votes for voters who have voted for, say, less than 3 games? That way we'd get rid of all the idiots who register under a new name every day and vote 1 on the single game they hate. It wouldn't hurt the honest "one time voters" either, since a review/score is only really relevant if you're able to compare it to other games anyhow. Just store the single vote in the database, but don't use it in the calculation of the game's total score. As soon as the registered name has voted for two more games you'll just add his/her score to the total. That together with the "more than 10 words in the review rule" would probably sort out the worst offenders. Perhaps..
_________________
Swedes visit NordicGamers for the latest game reviews in swedish! |
|
Wed Jan 21, 2004 2:41 pm |
|
|
Lintra
Elf Friend
Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES |
@Myrthos - Thanks for the data. I now see why we want to look at ... this is some of the most skewed data I've seen in ages (and believe me, I see a lot of crud doing what I do!):
From a total of 10,190 votes:
1. 866
2. 225
3. 225
4. 222
5. 354
6. 412
7. 610
8. 913
9. 1,466
10. 4,897
The rising cresendo of votes towards 10 is amazing! But then, there is the human effect involved (to some degree) if a person only feels so so about a game they tend to forget it and move on, never bothering to review it ... on the other hand a game really enjoyed, or really hated will be more likely to be reviewed.
@All the rest-
1. The 10's and 1's should be scrubbed after we set up the queue. The data is just too skewed. The 8's and 9's I can believe ... these are posted by enthusiastic people who just finished the game and really liked it, or are remembering a game they played many, many years ago through rose colored glasses ... there is most likely not much we can do about that effect.
2. I am VERY curious as to why there are essentially the same number of votes under 2, 3 and 4! This is a VERY odd coincidence!!
3. As to changes to the scoring system - I think losing the granularty going forward is a good idea ... reduce the rating system to 6 possiblilties. I liked goshuto's ratings:
10 - Greatest game ever.
9 - Definately worth the money and more.
6,7,8 - Good overall game.
5 - Average, wait for it to hit a bargin bin near you.
2,3,4 - Not recommended for those without masochistic tendencies.
1 - Do not under any circumstances buy this game! Not even if they're holding a gun to your head.
Except I'd change 5 to read average game, not bad. If you like the setting or the theme buy it, otherwise wait for it to hit a bargin bin near you.
.. but I will bow to what ever the group decides.
4. Let me know when, and how, I can help.
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless= |
|
Wed Jan 21, 2004 3:04 pm |
|
|
Toaster
Bread Alert
Joined: 27 Jan 2003
Posts: 5475
Location: Sweden |
I don't find it strange at all that the 10s is dominating the list, since many people just bother to vote on a game they really like.
I find goshuto's suggestion about the rankings best, but it'll only work good if all top and bottom reviews is monitored.
_________________
Tabbrowser Extensions
DictionarySearch |
|
Wed Jan 21, 2004 4:01 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
quote:
Originally posted by Lintra
@All the rest-
1. The 10's and 1's should be scrubbed after we set up the queue. The data is just too skewed. The 8's and 9's I can believe ... these are posted by enthusiastic people who just finished the game and really liked it, or are remembering a game they played many, many years ago through rose colored glasses ... there is most likely not much we can do about that effect.
THANK GOD! Somebody FINALLY repeated what I've been asking all along!!! You've got almost 5000 "Best Game Ever" votes. It's simply not possible, not accurate, and would need to be fixed if our ratings are to have any integrity. I'm not sure why nobody else could figure this out. Perhaps I need to take a refresher composition course...
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Wed Jan 21, 2004 5:16 pm |
|
|
goshuto
Wanderer
Joined: 29 May 2002
Posts: 1142
|
quote:
Originally posted by dteowner
I'm not sure why nobody else could figure this out. Perhaps I need to take a refresher composition course...
Guess now it's my turn to do it... 
quote:
Originally posted by goshuto
A rating of 10 accounts for ~48%. This means that (ignoring dishonest ratings), if we classify a rating of 10 as "best game ever made," roughly half of the users would have voted because they really, really liked a game.
*cough* 
_________________
"Tree stuck in cat. Firemen baffled."--Simcity 3K
"Life can only be understood backwards; but it must be lived forwards."--Soren Aabye Kierkegaard |
|
Wed Jan 21, 2004 5:23 pm |
|
|
dteowner
Shoegazer
Joined: 21 Mar 2002
Posts: 7570
Location: Third Hero of Erathia |
Touche, goshuto. 
So here's where it seems we're at. We can't keep 5000 "10" reviews in the database (what I've been saying all along), but we can't throw out anything (what Val's been saying, I think). Now, let's say we throw out all the "10" reviews that have less than 10 words of meaningful text. Let's say that gets us down to 1000 "10" reviews. Let's say that a mere 10% of those "valid" reviews come from users that have given "10" scores to more than 5 games. That means we have to send out 100 emails asking those users to fix their reviews because it makes no sense to have more than 5 games getting a "Best Game Ever" score from the same user?
This is why I said it MIGHT be better to start over. If anyone has a better way to address the problem (I think ignoring it would be a poor decision), I'm all for it.
_________________
=Proud Member of the Non-Flamers Guild=
=Benevolent Dictator, X2/X3 and Morrowind/Oblivion Forums=
Sorry. No pearls of wisdom in this oyster.
RIP Red Wings How 'Bout Dem Cowboys! |
|
Wed Jan 21, 2004 7:57 pm |
|
|
Lintra
Elf Friend
Joined: 23 Apr 2002
Posts: 9448
Location: Bermuda, the triangle place with SANDY BEACHES |
As a goof I went and looked at all the reviews I have posted ... man oh man do I wish I could change some of them!
For example: My 10 on M&M6 is totally undefended ... blank. But I stand by the rating.
An other example: M&M9 I gave a 7 or 8 to, I must've been in post let down shock. It did not deserve anything above a 6 (it wasn't THAT bad ... just a major dissappointment given how much I'd looked forward to it).
I gave out very few rankings below a 5 ... in fact I think I only gave out one.
Which begs the question:
@Myrthos - is it possible to edit one's own reviews?
Edited:
I just did a quick dump of my reviews and this is what I gave out as ratings:
code:
3 3
5 3
6 4
7 10
8 14
9 10
10 2
For an average rating of 7.4 ... but a large number of those I rated where very old ones that I remembered ... I do not remember the terrible ones 'cuz I never finished them ... or never invested enough time to remember them.
PS - The two 10's are for MM6 and Darklands so I don't feel I've been overly liberal with the 10's .... but I most likely gave out too many nines.
Looking at what I rated I basically had the following groupings:
1. Wish I'd saved my money. (<5)
2. Okay, but play once and give away. (5)
3. A good game, I enjoyed it. (6)
4. A really good game. Go back and play again. Recommend to friends.(7& some of 8 )
5. A great game ... almost a 6, but not quite. (9 and some of 8 )
6. One of the best ever. (10)
Looks a lot like Goshuto's groupings, eh?
_________________
=Member of The Nonflamers' Guild=
=Just plain clueless= |
|
Wed Jan 21, 2004 8:04 pm |
|
|
Gorath
Mostly Harmless
Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany |
:no. of 10s per reviewer:
This is irrelevant. We did not ask our readers to build a top ten list of the RPGs they´ve played. If somebody thinks he has to give 20 games a 10 it´s his own opinion. How many games are in the DB, 400? Where do you want to make the cut?
:emailing reviewers about the new system:
You must be joking. Nobody will be mailed. Just imagine 100 people really answer the mail and demand explanations. The system has to be changed in a way that every review(er) is treated equally.
_________________
Webmaster GothicDot |
|
Wed Jan 21, 2004 9:03 pm |
|
|
|
Goto page Previous 1, 2, 3, 4, 5, 6, 7, 8, 9 Next
All times are GMT. The time now is Tue Apr 09, 2019 8:46 pm
|
|
|
|
|
|
|
