Saturday, May 3, 2008

10/10

So, Grand Theft Auto IV received a plethora of 10/10 scores from various outlets around the internet and print. And my friends and co workers are up in arms. Pulling flaws from a seemingly endless bag, they deride the impossibility of a game with so many problems to be "10/10", with the subtext that 10/10 is perfect.

I submit that 10/10 should not be considered as perfect but should be considered to be one of the best games currently available. 5 star movies are not necessarily perfect, nor are five star hotels. The entire game rating system is so broken, basically decided by the American education system of 90-100 being an A, 80-90 being a B and so on. Where as a two and a half star movie would still be enjoyable to the people most likely to see that type of movie anyway, a 50 game is pretty much considered to be garbage. Game reviews are in this weird sort of range of 50-95, anything less than 50 is crap, anything 95 or above is fantastic. Less than 50, the ratings start to fall apart. I would like to see reviewers adopt the five star system or else use the alphabet grade system, where a C is mediocre a B is good, and an A is fantastic, and not use numbers with it at all, there's nothing less than an F, nothing higher than an A+.

So what's a five star game? An A+ game? For me, GTA IV certainly is. No one offers the GTA experience better than Rockstar. Call of Duty 4 is a five star game certainly. Mass Effect as well. Halo was probably four and a half stars for me. Saints Row was probably three stars.

There has also been a lot of talk about hype and brand, and I would really like to see a study done on whether a hyped game gets better reviews than an unhyped game. I think it's clear that brands make a huge difference, I think Mario gets an extra 5-10 review points just for being mario, and it's clear that GTA got an extra few points for being a GTA game. I'm sure had Saints Row 2 somehow eeked out the GTA IV experience, it would have gotten good reviews but I doubt it would have gotten many 10/10s.

So that leaves me in a weird spot. I don't really think GTA IV should be called a 10/10, but I disagree with the rating system as much as the score. I'm thinking GTA IV should be a five star game, and it should have some fellow games in that exalted list, just like films.

2 comments:

Ian McMeans said...

The rating system I've always advocated is turning review scores into a percentiles system - ie. "this game is better than x% of the other games we've reviewed".

This forces publications to be internally consistent (you can't have more than half your reviewed games have scores in the upper half), and has the side-effect of normalizing scores across publications (so that an IGN 90% would be directly comparable to a Times 90%, because it's forced to be equally rare).

Another nice side effect is that a "perfect" score of 100% doesn't mean flawless, it just means "as good as the best game we've ever reviewed". If a publication gives out too many of them, the scores will mean less... because if 30% of their games get a 5-star review (or 10/10, or whatever), that score from that publication spans the percentile range of 70-100%.

Another side-effect is that the publication doesn't need to cooperate - we can turn their scores into a ranking/percentiles. I wish metacritic or some other review aggregator would do this.

It doesn't fix the problem of bias (some publications favor some companies / franchises), but reweights scores to be equally meaningful.

Dave said...

Details of the rating systems aside, I think this whole 10/10 thing points out some serious flaws in the integrity of reviewers. Either having been susceptible to the hype machine, or selling out to fiscal interests (either direct from a publisher or indirect by traffic generated through high profile releases), there is clearly something fishy going on here. It's hard to look at a review score objectively when it's bordered on all sides by advertisements for the same product.

The sheer number of serious issues with the game and it's myriad mediocre mechanics should mean that at least something about them should have trickled into a review from one of these major sites. The fact that there is nary a mention of any of these issues leads me to believe that these reviewers were either encouraged to leave them out, or simply didn't play the game.

There is no way that a statement such as (from IGN) "The cover and targeting system work great. Blind firing with an RPG is a thing of beauty. Everything works in harmony and not a single one of the missions is bad. The most fun I've had in years." can even subjectively be observed as true.

The cover system is poorly implemented and is cumbersome even in the best of circumstances, and the targeting, assuming it is even aiming at the intended target is jarring. Everything works well enough together, but to say that not a single one of the missions is bad is maybe lie of the century. There are some god awful missions. GTA is a veritable cornucopia of "game design don'ts".