Flaw #1 with review scores: the 10-point scale

This entry is the first in a series I plan to write about why modern video game review scores are flawed.  One of the first steps in deciding whether or not to purchase a game is to read the reviews, and review scores are meant to provide at a glance what each reviewer thought of the game.  Converting paragraphs of text into a single numerical score, however, isn’t exactly a smooth process, and crucial information can be lost in translation.  But what good is a score if it does not accurately reflect the reviewer’s opinion?  In this post, I show why using a 10-point scale to review games is a bad idea.

The most critical problem with the 10-point scale is that the meaning of the middle scores is not well defined and not consistent from reviewer to reviewer.  It can generally be agreed that a game receiving a score of 9 or 10 is exceptional, and a game receiving a 0 or 1 (depending on how low the review scale goes, which also varies with the reviewer) is terrible.  But what can we say about a game that gets a 7?  Is a game considered average or below average if it gets a 5?  Admittedly, some reviewers and review sites do attempt to explain what their scores mean, but it would be nice to see a universal review scale so that such explanations would be unnecessary.

Another issue concerns the granularity of the scale, or how finely the scale is divided.  For example, a 10-point scale with granularity 0.5 allows scores of 0.0, 0.5, 1.0, and so on, all the way up to 10.0.  If a scale has too little granularity, then the review score has little meaning since two games could get the same score even if, comparatively, one is noticeably better than the other.  On the other hand, too much granularity goes beyond the realm of human discernment – is a game that earns a 6.3 really any better than a game earning a 6.2?  There’s also the problem of having constant granularity along the entire scale.  We may care whether a game earns an 8 or a 9, but we’d probably care a lot less whether a different game earns a 2 or a 3.

Given these points, I would argue that a traditional letter grading system would be best, provided that the reviewer uses a different mindset when assigning a grade and does not equate the letter grades with numbers.  Everyone can intuit that A = superb, B = good, C = average, D = below average, and F = failing, and with the addition of +’s and -‘s, the granulation is ideal.  I’ve noticed that EGM magazine and 1up.com have switched to a letter-grade review system, which is a good start.  Gamespot also revamped its review system not too long ago, and their review scale now has granularity 0.5 (as opposed to 0.1).  These are all changes for the better, and it would be great if other reviewers followed suit.

Advertisements

0 Responses to “Flaw #1 with review scores: the 10-point scale”



  1. Leave a Comment

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s





%d bloggers like this: