RPGDot Network    
   

 
 
Redmoon Online
Display full image
Pic of the moment
More
pics from the gallery
 
 
Site Navigation

Main
   News
   Forums

Games
   Games Database
   Top 100
   Release List
   Support Files

Features
   Reviews
   Previews
   Interviews
   Editorials
   Diaries
   Misc

Download
   Gallery
   Music
   Screenshots
   Videos

Miscellaneous
   Staff Members
   Privacy Statement

FAQ
Members
Usergroups
Skewed Reviews? @ NWN2News
  View previous topic :: View next topic
RPGDot Forums > News Comments

Author Thread
Dhruin
Stranger In A Strange Land
Stranger In A Strange Land




Joined: 20 May 2002
Posts: 1825
Location: Sydney, Australia
Skewed Reviews? @ NWN2News
   

Rhomal from NWN2News dropped us a line to say they have a new editorial online, this time on whether major sites have <a href="http://www.nwn2news.net/modules/smartsection/item.php?itemid=18" target="_blank">skewed reviews</a>:<blockquote><em>This got me thinking, looking at the major online PC gaming sites, I am hard pressed to recall more then a couple games within the recent past months that had a below average rating. Now unless we are going though a new golden age of PC gaming shouldn’t it be quite difficult to get a rating above 75%? More then 75% indicates to me a way above average game. A game with new ideas, a fresh approach to old ideas or just fantastic game play. A game in the top 25%, and especially top 10% (90%+ rating), should be reserved for the elite few in my opinion. And I am pretty sure I am not alone in thinking a 80%+ rating shouldn’t be handed out like candy.</em></blockquote>
Post Fri Dec 02, 2005 9:04 am
 View user's profile
abbaon
Guest






   

That complaint just never gets old. Hey Rhomal, if you guys need a target for your next rant, you might like to take a shot at painting-by-numbers editorials.
Post Fri Dec 02, 2005 9:12 am
 
FrancoTAU
Village Dweller
Village Dweller




Joined: 05 Oct 2005
Posts: 5
   

Although he isn't breaking any new ground, i do like the statistically breakdown. Pretty funny that IGN and Gamespot rate half the games as 80% and above. I wonder what the numbers would be if you took budget titles out of the picture.
Post Fri Dec 02, 2005 1:38 pm
 View user's profile
Wildefire
General Goods Vendor
General Goods Vendor




Joined: 03 Oct 2005
Posts: 1
Location: USA
   

It is widely accepted that the bigger gaming sites tend to rate on the high side, but if you know a site's tendency, and you know how much you agreed with their previous reviews (the reviews, not the scores), you can often still get some meaning out of their scores and reviews by adjusting them with your own 'handicap' for that site.

I would like it if all sites and magazines clearly stated their reviewing policies. I've noticed a couple of the print mags doing it recently, and I hope this continues. Things such as whether they only review final games, whether more than one person has a say in the final score, and whether the reviewer is required to actually finish the game (or in the case of MMOs, minimum level reached or hours spent) can all affect how accurate and reliable a review will be.
_________________
For each toon kills the mob he 'groes, yet each toon does not die.
Post Fri Dec 02, 2005 3:42 pm
 View user's profile
niteshade
Keeper of the Gates
Keeper of the Gates




Joined: 09 Jul 2005
Posts: 100
   

I don't think most people associate a grade of 50% with average. In school, 50% is a failing grade and 80% is only decent. I believe that is the system most people rate games on.
Post Fri Dec 02, 2005 3:53 pm
 View user's profile
mogwins
Village Leader
Village Leader




Joined: 15 Jan 2002
Posts: 99
Location: Wales
   

quote:
Originally posted by niteshade
I don't think most people associate a grade of 50% with average. In school, 50% is a failing grade and 80% is only decent. I believe that is the system most people rate games on.


But your example relates to a physical quantity - e.g you got 8 out of the 10 answers correct, thus 80%. I don't think anybody really believes that most games are 80% perfect?

Rating all games between 80% and 99% just seems pointless - why not "up the contrast" and normalise these numbers so that they fall between 0 and 100?
Post Fri Dec 02, 2005 4:01 pm
 View user's profile
niteshade
Keeper of the Gates
Keeper of the Gates




Joined: 09 Jul 2005
Posts: 100
   

While that is true, I still think people are hardwired by the academic system to think of percentages in those terms. If you actually ready the reviews, if a game does get a 50% the review is usually pretty consistant with an F. While if a game gets a 70% it's consistant with a C-. I think for most people, that's just the most intuitive.

Plus many would say that if a game does do 50% of everything wrong, that would be grounds for a failing grade.
Post Fri Dec 02, 2005 4:31 pm
 View user's profile
mogwins
Village Leader
Village Leader




Joined: 15 Jan 2002
Posts: 99
Location: Wales
   

quote:
Originally posted by niteshade
While that is true, I still think people are hardwired by the academic system to think of percentages in those terms. If you actually ready the reviews, if a game does get a 50% the review is usually pretty consistant with an F. While if a game gets a 70% it's consistant with a C-. I think for most people, that's just the most intuitive.

Plus many would say that if a game does do 50% of everything wrong, that would be grounds for a failing grade.


Fair 'nuff.
Post Fri Dec 02, 2005 5:23 pm
 View user's profile
Ectoterrestrial
Guest






its academic
   

I agree. I think the school grading system is engrained into the American psyche from which many game reviews are produced. Reviewers elsewhere are forced to operate within a similar construct.

That being said, I would like to see a breakdown of review scores by month of the year in which a game is released. Are reviews higher scoring in November and June, (Holidays and end of school year) than say, July and February? That might show the trend of market forces more clearly, although it wouldn't distinguish cause and effect.
Post Fri Dec 02, 2005 6:18 pm
 
Amelia
City Guard
City Guard




Joined: 18 Jun 2004
Posts: 141
Location: Ong's Hat, New Jersey
   

For the most part, I think numerical scoring systems are too ephemeral and generally useless for any sort of objective review and I eagerly await quality magazines or review sites that won't find them relevant.
Post Fri Dec 02, 2005 8:53 pm
 View user's profile
Gorath
Mostly Harmless
Mostly Harmless




Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany
   

quote:
Originally posted by Wildefire
It is widely accepted that the bigger gaming sites tend to rate on the high side, but if you know a site's tendency, and you know how much you agreed with their previous reviews (the reviews, not the scores), you can often still get some meaning out of their scores and reviews by adjusting them with your own 'handicap' for that site.

I would like it if all sites and magazines clearly stated their reviewing policies. I've noticed a couple of the print mags doing it recently, and I hope this continues. Things such as whether they only review final games, whether more than one person has a say in the final score, and whether the reviewer is required to actually finish the game (or in the case of MMOs, minimum level reached or hours spent) can all affect how accurate and reliable a review will be.


The problem is much , much bigger. Read this blog post and tell me if one can come to any other conclusion than that professional (= paid) reviews are faulty by definition. A fair review under such circumstances is nothing more than a coinincidence.
_________________
Webmaster GothicDot
Post Fri Dec 02, 2005 9:24 pm
 View user's profile
Wolfie_
Guest






   

I like what Wildefire mentioned, a declaration of review process / policy. Personally, I don't give much credence to a gaming "score" (like a game is scored 85%, 85% of what? Does 100% mean the perfect ultimate game? Doesn't exist, the games industry is a continually evoluting process). Often by reading the words in a review, you can get a sense of what a reviewer's "stand-point" is and so can then understand why a particular score has been given. However, if a formal critera is layed oit, this would have the two-fold advantage of informing the reader how a score was reached, and would (hopefully) lessen the reviewers own personal bias.

Speaking of bias, one wonders how much of that bias is affected by the volume, style and sheer "loudness" of press releases, media meterials and constant "in your face" advertising generated by game publisher's marketing departments? An old saying goes: "the squeaky wheel gets the oil", the same could be said to apply to game reviewers too, they they are handed lots of sales litter ... err sorry, I mean sales literature, it makes the lazy reviewer's life easy, they can just quote the sales pitch (maybe even go to the effort of changing a few paragraphs around, a word here and there), whereas plenty of great games have come and gone without much of a blimp on many game reviewer magazines / web sites because there was not enough attention given (the original Gothic springs to mind outside of Germany and Europe generally).

[/i]
Post Fri Dec 02, 2005 11:38 pm
 
Guest







   

Just for fun, a quick graph of the number of times RPGDot has awarded each score. My explanation: we're going through a golden age of PC gaming.
Post Sat Dec 03, 2005 12:23 am
 
Gorath
Mostly Harmless
Mostly Harmless




Joined: 03 Sep 2001
Posts: 6327
Location: NRW, Germany
   

You can´t compare this to a professional publication. Our scores represent only the reviewer´s opinion. More often than not they can´t even be seen in the context of similar games. Big publications have editorial supervision.
_________________
Webmaster GothicDot
Post Sat Dec 03, 2005 1:01 am
 View user's profile
Wildefire
General Goods Vendor
General Goods Vendor




Joined: 03 Oct 2005
Posts: 1
Location: USA
   

quote:
Originally posted by Anonymous
Just for fun, a quick graph of the number of times RPGDot has awarded each score. My explanation: we're going through a golden age of PC gaming.


Har, I was going to quip that "Well that's because we never reviewed Dungeon Lords-" but we did! 8 out of 10? Wow. I think that's a record for that game! Seriously though, when looking through the list of RPGDot reviews, I don't see very many games that really even merit low scores in my opinion (Dungeon Lords excepted). This is because we tend to rate the games we're actually interested in playing... if a game looks like it's going to be total crap we probably won't touch it. Magazines and professional major sites try to review everything that gets released by a major publisher though (especially during dry seasons), and they'll encounter more real stinkers than we will as a result.

But like Gorath said, when you see a review here, it's the reviewer's honest opinion; not some externally-influenced propaganda or half-baked puff piece that's not even based on gameplay. If we call it a "review" then it's based on a complete play-through of the game (or, in the case of MMOs, based on enough gameplay to voice an informed opinion). Otherwise it's an 'impression' or 'preview.'[/i]
_________________
For each toon kills the mob he 'groes, yet each toon does not die.
Post Sat Dec 03, 2005 1:37 am
 View user's profile


Goto page 1, 2  Next
All times are GMT.
The time now is Thu Apr 11, 2019 7:09 am



Powered by phpBB © 2001 phpBB Group
 
 
 
All original content of this site is copyrighted by RPGWatch. Copying or reproducing of any part of this site is strictly prohibited. Taking anything from this site without authorisation will be considered stealing and we'll be forced to visit you and jump on your legs until you give it back.