The Alphaville Herald

Essay: The ESRB game rating system is broken

by Alphaville Herald on 03/02/04 at 7:19 pm

By J

I recently wrote an article for the Alphaville Herald on how I believed that Electronic Arts had misled the Electronic Software Rating Board (ESRB), as well as consumers and parents alike, about the rating of their popular MMORPG, The Sims Online. My criticism was met with a variety of comments from readers of the weblog where the article was posted. Some agreed with my position that EA has not given an accurate account of The Sims Online when it was rated. Others believed that the ESRB is at fault and that the game rating system is ineffective when it comes to MMORPGs. In any case, most who posted comments agreed that something about the process needs to change. After considering these comments and after studying the situation carefully, I believe that both EA and the ESRB need to reconsider their actions when it comes to rating online roleplaying games. In this essay, however, I would like to focus on the ESRB and explain the flaws of the current system. Further, I will make some suggestions for how it could be improved.

Static and Dynamic Game Environments

To begin with, I would like to explain how the dynamic environment of an MMORPG, such as The Sims Online, is substantially different from the static environment of many other games. The classic game Tetris is a good example of a static environment. A player’s interaction with the game is limited by the game’s programming. For instance, a player cannot create a new piece that was not originally programmed into the game or create things less perceptible like atmosphere and culture. This type of game is simple in the sense that the structure and potential outcomes of the game are fixed. This is even consistent with more elaborate games, like Doom. A Doom player is still limited by programming to where he or she can go and the other characters that he or she interacts with.

MMORPGs, like The Sims Online, are dynamic game environments. They are highly complex because they involve other real people who interact with each other. The complexity is even further increased because the characters are not fixed, making the game highly unpredictable. In The Sims Online for instance, the game programming does not determine what type of property a player chooses to create (they may choose to make their property a house for friends, a pet store or cyber-brothel). MMORPGs adopt dynamic qualities that static games cannot. They adopt cultures, governments, personalities, friendships and rivalries, just to name a few documented examples. These features are impossible to predict by the game publisher. They are also impossible to observe at any one point because they are always evolving and changing. Therefore, an MMORPG can never have its contents evaluated with the expectation that this evaluation will remain the same at any later period in time.

The ESRB Rating System

Next, I would like to examine the rating process of the Electronic Software Rating Board. The council uses what it calls “the rating pool” as their experts for judging games. The rating pool is a group of freelance individuals that ESRB hires and trains in the art of rating games. They are expected to have no ties with the video game industry and their identity should remain anonymous.

The rating process is conducted before the game is released and distributed. The game publisher, like Electronic Arts in the case of The Sims Online, must file a questionnaire and submit a video of the game. The questionnaire and video footage must provide “an accurate representation of the context and product as a whole.” The videotape must show “the most extreme content that can be achieved” in the game. Three members of the rating pool then review the game and they must come to a consensus on what the game should be rated. If they cannot come to a consensus, more reviewers may be utilized.

The ESRB Rating Process and Game Environments

The ESRB rating system has been designed for a static environment, in which I believe it does an efficient job of rating games. With simple games, such as Tetris or Doom, a questionnaire can determine the fixed content of the game. Video taped footage can meanwhile show the context and product as a whole in a way that is most likely sufficient for raters. These two criteria may be enough for static game environments and may not require the reviewers to directly interact with the game at all. However, this process poses a variety of problems when it comes to evaluating MMORPGs.

As I described dynamic environments, they are unpredictable. For instance how does a game publisher answer the question, “what is the extreme content of a game when you do not know what content will be achieved?” Electronic Arts cannot begin to assume what the most extreme content will be due to the fact that they cannot predetermine the actions of future players. They do not know who will purchase the game or how they will interact with other players. As soon as the game is on a store shelf, it is out of the publisher’s direct control.

I would like to use The Sims Online as a detailed example of this phenomenon. In The Sims Online, players are allowed to construct their own avatar, or graphic representation of themselves in the game. In some ways, the character that one chooses is fixed. There are a specific number of different body styles and a preprogrammed number of heads. If just matching a body to a head was the limit to the game, Electronic Arts could provide a mathematical figure for the most extreme content, i.e. there could only be one girl with red curly hair and a blue dress. But online roleplaying games are not structure to fit inside the contents of preprogrammed limitations. Online roleplaying games facilitate interactions between minds. Even though there are a preprogrammed number of different body styles for characters in The Sims Online, two people can take the exact same girl with red curly hair and blue dress, but make them radically different because one interacts as a pet storeowner and the other interacts as a prostitute. Electronic Arts cannot program the way that these characters interact (in the sense of their attitudes, morals, beliefs, etc.) because these interactions transcend the borders of the game.

The comments of some people on the Alphaville Herald weblog have argued that it is not that rating of the game that is at issue, but rather it is the players that should be rated. This is correct, but also misguided. True, the players are the problem with the game. They are the unknown variable that causes the game to diverge from its intended programmed structure. However, The Sims Online still facilitates the interactions of these players. Electronic Arts has created a game with enough freedom to introduce new ideas and goals. Electronic Arts needs to be aware that interactions from outside the borders of the game will enter and likewise the rating of the game needs to take this freedom into account.

Three Flaws of the Rating Process

I believe that there are three important flaws to the ESRB rating system when it comes to online roleplaying games. These flaws are the fundamental criteria that ESRB uses: the rating pool, the questionnaire and the video footage. I will explain why these criteria are inadequate and how they may provide an inaccurate rating of the game or allow the publisher to mislead the rating board.

An online roleplaying game draws participants from all over the world and all walks of life. They are all real people who have different moral backgrounds, different lifestyles, different codes of law and an immeasurable number of other variables that they will bring to the game. If the rating pool were to give an accurate account of how this dynamic game should be rated, the pool would have to be a cross section of the people who play it.

The ESRB draws freelance individuals from the Manhattan, New York area. They must have a flexible schedule that allows them to come to the ESRB office 1-4 times per month. I would like to argue that unless there are additional measures to control the demographics of the raters, this will not yield a statistically sound cross-section of the population. The fact that all of these people have flexible, freelance schedules and live in the Manhattan area cuts down on the diversity of the pool. Further, if the rating is intended to inform not just the players, but also parents, this rating pool should consist of a large portion of parents. However, the demands of a flexible, freelance schedule in the Manhattan area does not seem very accommodating to draw a large amount of parents.

It would be ineffective to try to understand the content of dynamic a game by using a standardized, static questionnaire. ESRB says “publishers fill out a detailed questionnaire explaining exactly what’s in the game.” By this method, the questionnaire can only explain what is in the game before other players join. Since the game does not actually begin until the game is distributed and players have entered, this is an insufficient process. This may even allow the publisher to downplay what the presupposed extreme content may be and therefore mislead the ESRB and the consumer.

Finally, the ESRB system for rating online roleplaying games is flawed because game raters do not actually play the game, but rather they watch a video produced by the game publisher. This may be effective for simple games where the publisher can show the limited number of moves or levels and show the final goal, but complex online roleplaying games cannot be treated this way. The publisher on an online roleplaying game cannot show video footage of things like culture, attitude, friendships, enemies or love. They cannot show how these things evolve over time and what course they may take. It is impossible give an accurate account of an online roleplaying game’s content without personally interacting in the game.

Current Ineffective Strategies

In the Alphaville Herald weblog, there was a strong opinion of many players that the current ratings of online roleplaying games are ineffective. The ESRB is aware that dynamic roleplaying games will evolve with the players that join and they have adopted a disclaimer to try and explain this. As ESRB says, “Additionally, online games that include user-generated content (e.g., chat, maps, skins) carry the notice ‘Game Experience May Change During Online Play’ to warn consumers that content created by players of the game has not been rated by the ESRB.” The mere statement “Game experience may change during online play” is not explanatory of what may occur in online roleplaying game.

In The Sims Online, for instance, the most extreme thing you can do in the static state of the game is to “Kiss hotly” or lay in a “love bed.” These things may not be very extreme by ESRB standards and may be suitable for minor. However, the dynamic environment of the game has given rise to more provocative content. For instance, widespread pornographic and sexually deviant behavior have taken root and allowed children as young as 13 to participate in and be exposed to it. The extreme elements of the dynamic game are much different from the extreme elements of the static game, but both exist within the game.

In my opinion and the opinion of many players, there should be a new, more effective strategy for explaining that these games may take new directions that are extremely different from what the static environment of the game was intended for. For instance, a new rating of “ORPG” or Online Roleplaying Game may stand by itself or accompany the traditional age rating of the ESRB. A more detailed disclaimer should be utilized to provide better information to parents and players. For instance, “Game experience can and may change drastically over time. Strong violent and sexual themes may develop throughout the game. Player assumes all risk as to what they are exposed too.”

Suggestions for an improved rating system

There is definitely a need for a revised system of rating online roleplaying games. Even if some changes are put into place, they will need to be open to more improvement in the future as the online roleplaying game industry develops. In any case, I would like to conclude by making some suggestions of ways in which the ESRB can improve this system.

First of all, a new rating for online roleplaying games should be added, as I detailed in the previous section. It should have a better explanation of how these online roleplaying games may change and the kind of material that may develop (i.e. violent, sexual, pornographic, cultic, etc.). It should also list the age of the audience for which the game is best suited.

Second, game raters should actually play the game. Dynamic game environments cannot accurately be viewed be external means. An even more efficient system would be to hire and train raters who have played the game for significant period of time where they have become accustomed to the game and immersed in its culture.

My third suggestion is that more people to rate online roleplaying games. The rating pool should also come from a more diverse background. A cross-section of the rating pool should reflect a cross-section of the people that will actually play the game. The intended audience of the rating should also comprise a large portion of the pool, i.e. teenagers and parents. This would provide a broader review from wider range of opinions.

The content of the online roleplaying game should be audited annually or semi-annually by the ESRB. This would allow the game rating to evolve along with the ways in which the game changes. This would keep publishers accountable for their game content beyond the original programming. Finally, the ESRB should reissue the game rating if the original rating has been violated. Publishers should be forced to change their programming or advertising if a rating change is necessary.

Conclusion

The software industry is continually evolving and the ESRB needs to follow suit. The online roleplaying game segment is rapidly growing and therefore it will continue to affect more and more people. Substantial changes need to take place if these games are to be accurately represented and played by minors.