Ekim\'s Gamer View: Review: Impossible
Ekim, 2003-06-06

It's good to be back. It took me a while to get up to speed, but I quickly realized that I didn't miss that much while I was away. Not in terms of CRPG releases anyway. In my free time while on vacation, away from any type of gaming, I got to reflect about a few things. My perpetual love/hate relationship with MMORPGs for one thing. Since my time off saw the birth of our sister-site MMORPGDot, I thought I'd kick off my return with some thoughts about this always evolving genre and how it effects reviewers.

Your mission Mr. reviewer, if you accept it…

So, my thoughts about the genre mainly focused around the actual reviewing of the games of this complex genre. Actually, I debated the very relevance of MMORPG reviews. Most single-player games ship with all their features completed and polished. There are exceptions of course, and most single-player game that ships with incomplete or buggy features gets trashed by the reviewing community. It's normal, we rightly expect nothing less than completed and thoroughly tested software when it gets to store shelves. MMORPGs are a different crop, however. I don't think I can say of any previously released MMORPG that they have ever shipped entirely completed.
Whether that fact is a positive or negative one is for another debate. MMORPGs grow continuously, they never stay quite the same as the day they launch. In fact, I dare anyone to point me to a MMORPG that plays exactly as it did when it launched a year after its release. Whether it's to correct some anomalies, fix bugs, or add content and features, MMORPG patches are a fact of life, a necessity. Now, should this be taken into account in a review, or should the game be judged as it is at launch? Is it fair to staple a game with a good/bad label so early in its life? Should reviewers keep in mind that the game they are reviewing will change many times even soon after they send their text to be published? Should MMORPGs even be reviewed at all?
I've reviewed two MMORPGs for RPGDot so far. I've given them generally fair scores, and I have said of both that they were more tailored towards newcomers to the genre. Not being a newcomer myself, I stopped playing them soon after their review was completed. Should I log back in both these games today I am sure I would have a different experience than when I first tried them. Both the original comments and scores would most probably change, for better or worse.

You've never seen me very upset

When Anarchy Online (AO) came out, I bought and played the game for about three months. My first reaction (within the first two weeks) was that the game was exceptional and highly rewarding, even though buggy. The next two weeks served to tone down my first impressions as the tedium began to sink in, but I was still having lots of fun with it. Had I reviewed it then, after a month of play, my score would probably still have been between 75% and 80%, a fair score given that the game was laden with bugs and many technical problems. Three months later though, I was ready to give the game a good trashing, and a score of about 40%, nothing more! Last fall I decided to try out the game again, using their promotional 14 days of free gaming for old players. The game seemed stable by then, and the errors made a year before all seemed to be fixed. Still, I wouldn't have given it a higher score than 65% to 70%, and I quickly moved on, canceling my account just before the 14 days were over. The point is that by then my comments regarding the game would have been completely different than just a few months earlier.

Reviewing a game is like taking a snapshot of someone at any given point in time. Comparing the picture of a 40 year old man to an older picture of himself taken 10 years earlier, there's a good chance that everyone could say that they were both of the same person without even knowing him. Yes, the hairdo and clothing might have changed, but that person's features would still be recognizable and pretty much the same as it was on the older picture. Single-player games are very much like that: they are released as mature, adult products that won't change very much anymore. A MMORPG would be more like taking the picture of a 6 months old child. Another picture taken even just a year later would show a huge transformation. Let ten years pass and there's a good chance that you wouldn't even know that the two pictures were of the same person.

Let's settle the score

So how can a MMORPG be properly (and fairly) scored? Should we make a monthly, or quarterly review of it? A MMORPG review remains just a snapshot of a young game that is bound to change drastically in the following months. How can a reviewer even take that into account successfully without coming back to the game from time to time? How can a reviewer make it clear that the score he gives now can never be definite or even quite accurate?

I've read lots of recent reviews of MMORPGs lately which started with a cautionary notice that generally went like this : "This game was reviewed as is, and there is no way to take into account how it will turn out in six months." It's crude and ungraceful, but true and absolutely correct. It proves that reviewers are beginning to realize that reviewing a MMORPG is some very dirty work. There's bound to be a smart mouth sending him an email six months after it has been published to tell him what a jerk he is for giving the game such a low/high score. How can we foresee where any MMORPG will be six months down the road? I'm not even sure there is a way to more accurately rate MMORPGs, but if anyone else does, please let us know and let's discuss it in the forums.