Polygon, Sim City, A Review, and the Implosion of the Internet

Everyone is fully aware of the embarrassing launch that Sim City has recently undergone. And while some talk is about EA and their ineptitude of having servers ready, the future of always-online games, and the overall frustration gamers normally vent; Polygon decided to open a new can of worms with Sim City and wrote a review. The review itself wasn't the problem, it was what they did after the review was written that caused a backlash.

The above score of 9.5 was the written review of Sim City before it's initial release. As you can see, it was a very highly regarded game and almost reached a perfect score. This was on 3/4/13, the day before release. Less than 24 hours later, it received an updated score:

Still a pretty good score, but why the change? As everyone is well aware, Sim City has...to put it lightly....botched their release day. The game is nearly unplayable to most gamers. A game being unplayable is a pretty big problem, and I would say warrants a worse score than an 8. That is if you decide that server problems like this should influence the score. Which it did...to an even greater extent:

This is the most recent score that Sim City has on Polygon. A 4. Within two days, a near perfect game is ranked as a game you should ignore. And the biggest reason is the server issues. Which leads me to the main point of this blog.



Perhaps it's because I'm not in the business, but I have no problem with a game getting a slight ding against it due to online issues. If a game has online problems, people who read the review need to be made aware of it since they will be throwing money at the game to play it. But perhaps online instability shouldn't be a large ding against the game on a review. Thanks to where we are in gaming, patches and fixes can come on a daily basis. What is broke today, may be fixed tomorrow. But how does a reviewer attack that exactly? Clearly server problems weren't evident before the game release which would warrant the score of 9.5. There are three ways I think this should be approached, both without touching the final review score:

1. Review Game but Update with Notes - This would be for reviews that come out before launch day. Which is the case for most big time games. Usually online problems are not persistent for review (and early leaked) copies of the game. It either works or it doesn't. Score the game based on what it deserves at that time. However, if the game releases and is unable to work online, you don't update the score. You update the REVIEW, and you do it at the top where everyone sees it immediately. Don't hide it in the original review and don't create a different section at the bottom. Let the readers be aware that there is an update to the review that doesn't go into the final factor of the score, but should go into the final factor of your purchasing decision.

2. Hold the Review Until Launch - Blasphemy. Sites need these reviews for unreleased games mainly for hits which brings in the money. This is definitely the worse of the two, but it can be considered. The problem is the competition with other gaming sites and who releases the reviews first. People care less about the accuracy of their reviews and more about how many views their review receives. This could be true, or it could be that they do care about their reviews but the higher ups don't. This would lead into discussion about embargoes, and that's not something I'm going to touch. Especially since I'm not too familiar with them.

3. Review Online Mode Later - While this wouldn't work with Sim City due to it always being online, some games have a functional single player mode and a broken multiplayer. In cases like this, the single player can be reviewed but leave a discretion to the reader that the multiplayer can't be reviewed yet because it's broken. Anyone who played Brink on the PS3 on launch day remember that you couldn't play online due to the Sony hack. In this case, review the single player and let the readers know that the multiplayer will be reviewed at a later date. Same can be said for any multiplayer that doesn't work immediately.

Polygon however decided to take their own route on it, and just change the score with a reason as to why. But I have a slight problem with it. For their score of 8, they listed that being unable to play the game was a big reason. Understandable, but if that was a huge concern, you would think an unplayable game would receive an unplayable score. Fast forward one more day and you receive more problems that stem from the same issue of server overload. So they lower the score again. But while you can actually get into the game, there are other issues such as the speed of which you can play it (no more "Cheetah" speed), crashing issues and no leaderboards. That caused the game to receive a 4. Maybe I'm a crazy person, but I would much rather play a game with some issues than not play a game because of issues. But scoring a game less because it actually works and still has problems as opposed to not working in general is completely ludicrous, and the reviewer should rethink the approach.

Many seem concerned with Polygon's approach to changing games. But it shouldn't be that big of a shock as it is mentioned in their review guidelines:

Taken from Polygon:

Polygon's reviews and database have been built based on the idea of updates, or "bumps," as I've called them. If a game changes in a substantive way, we can add an update to our reviews that informs you how and why, and we can modify our scores accordingly. This will appear on the reviews in question as a timeline of that game's evolution and our corresponding recommendation (or lack thereof). The original review score will never vanish or go away, but our readers will be able to better understand where our opinions as a site reside over time for games we review.

We do not guarantee that we'll be able to do this with every game, and whether we do so or not is solely at Polygon's discretion. We will act in what we consider the best interest of our audience, while being as fair as possible to the developers and publishers who pour time and money into the games you play. If a game sees substantive improvements that make for a better experience, we want to reflect that. If a game is less worthy of your limited time, we also want to reflect that.

 Whether you agree or disagree, they have it on their site as to their approach on reviews. But one should question exactly what it would take for them to update their scores. Games have problems with launches. It's common. Everyone is well aware of similar issues Diablo 3 had upon it's launch, but it received a perfect score. What is weird though...it's online is STILL messed up....It hasn't been updated, it never received a docked score, and was one of the biggest releases last year. This idea of pick and choose what reviews will be updated is not something a professional site should use, even if I agree that a game worthy of your time should receive this attention. Clearly, it doesn't. It only matters based on what will create hits at that time. Even the creator of Minecraft made comments on it because he constantly makes updates that improves the game. Oh, and in case you haven't heard, Minecraft is a pretty popular game. Not just on PC, but 360, Android, and IOS.

Polygon has made some bad decisions in this review, and to make it worse, they don't even follow their own rules. Here is what it says about a 1:

A game that doesn't properly function, which is what Sim City was. Instead of giving it an 8, it should have received a 1. Bottom line based on the review guidelines that the site itself set up. If they don't follow their own guidelines for scores, what makes Polygon a site worth trusting with reviews? Back that up with their pick-and-choose approach to what games will get updated scores, and I wonder why anyone would go to Polygon for their reviews.

Despite this being a Polygon centered blog questioning their review not only for Sim City but review scores in general, I think it sets up a precedence for the possibility of future online-only games. But it's not only online-only games that are affected. Every Call of Duty or Halo release seems to cause problems on Xbox the first day or two due to sheer number of people online at any time, and I don't think there's been a single MMO that hasn't released without a hitch. It's not singled out to Sim City. But Polygon needs to think about their review process a little bit more before this happens again, because the Sim City review has painted them in a negative light. Especially if they don't treat this game equally to other games with similar problems on launch day.