NEWS

Publishers Overrating The Importance Of Reviews

Tom Ivan's picture

By Tom Ivan

November 26, 2009

See also:

Related Articles:

Game review scores are one of the least important factors when it comes to making purchasing decisions, according to the findings of a new survey by Cowen and Company.

“Videogame management teams have become increasingly focused on their ability to deliver highly-rated titles, as measured by online critic aggregator sites such as Metacritic and GameRankings” said Cowen analyst Doug Creutz, reflecting on the results of the firm’s Fall 2009 Video Game Survey, which polled 1,312 individuals and 1,013 game players on the subject of purchase intent.

“We note, however, that critic aggregator scores were judged by our survey respondents as being the least important factor out of eight categories in deciding whether or not to purchase a game.”

Respondents were asked to identify the most important factors they consider when choosing which games to purchase. Using a scale of 1 (very unimportant) to 5 (very important), the genre of a given game was deemed to be most important, earning an average rating of 4.20.

The second most important factor, how much people liked an earlier series entry, earned an average rating of 4.06, demonstrating the value of strong game franchises. The third most important factor was the price of the game (rating of 3.82), followed by word of mouth about the game (3.70), how the game looked when seen in store, online or in advertisements (3.55), and feelings about the publisher’s previous titles (3.33). Rounding of the list were critics’ reviews of the game (3.19) and scores on review aggregator sites like Metacritic (3.11).

“Word of mouth received a much higher score than critical aggregator rating. Thus, we believe that while Metacritic scores may be correlated to game quality and word of mouth, and thus somewhat predictive of title performance, they are unlikely in and of themselves to drive or undermine the success of a game,” said Creutz. “We note this, in part, because of persistent rumours that some game developers have been jawboning game reviewers into giving their games higher critical review scores. We believe the publishers are better served by spending their time on the development process than by grade-grubbing after the fact.”

Elsewhere in Cowen’s report, Creutz suggests that the “Wii bubble could be deflating” and “PS3 may finally be poised for a rebound”.

DubsTF's picture

Wait, this is the same buffoon who recently published the latest installment in the ongoing "Nintendo is doomed" saga?

Next story.

savagehenry's picture

Back in those dark days before the internet we were reliant on the games magazines like Edge to deliver us a guide of what to watch though their previews and interviews. There was no reveal trailers available no lengthy developer diaries that sometimes get update on a weekly basis, just the pages of your favorite magazines. I don't think the method of reviewing has changed greatly to accommodate the wealth of information that we now have at our disposal.

Reviews become just a clarification of the things you probably already knew. A high score doesn't necessarily mean that a game is going to be palatable to a mass audience, you have to take individual tastes into account.

Gamers can now follow games through early stages of development right up to release, large publisher make it there business to court the gaming media so that their product is shown in the best light and by the time it reaches the review stage, we gamers usually have a pretty good idea what a game is all about, we've most likely already made a choice about whether we are going to purchase it.

Alex_V's picture

One weakness of the question asked (in terms of the conclusion drawn) is that it fails to account for game review score's influence on other factors. A genre can be helped to be made popular through critical acclaim. The influence of a previous entry in a franchise can be review-related influence. Word of mouth is undoubtedly influenced by critical reception.

This is also a measure of what people think the factors are in their buying decisions. Rather than signifying that review scores are of low importance, it may simply signify that people underrate how influential they are.

poisse's picture

(ok I'm a game reviewer but still) this research ignores the fact that games are the only cultural industry where critical acclaim corresponds with commercial success: uncharted 2 being a great recent example. Now compare with what music or films make the most money; ergo games reviews are very important!!!

hansg's picture

While I understand your conclusion from your own perspective, I don't quite think you've proven causation yet. Do strong reviews cause strong sales? Or do strong sales come from other sources, and are strong reviews of such games merely indicators of reviewers being in touch with the market at large? (not that that is a bad thing, far from it)

As the article states, the reality is that many factors come into play when deciding to buy a game, and reviews are just one of them. To give some examples of my own buying decisions: I bought Half Life and Deus Ex largely on word of mouth, and the sequels because I had good experiences with those games - but I did wait to see if they picked up at least halfway decent reviews.

I loved Far Cry, but largely negative reviews stopped me from buying Far Cry 2, and largely positive reviews couldn't get me to buy Crysis (I didn't enjoy the demo, so I didn't buy the game). Then again, average reviews didn't stop me from getting Stalker (and I thought it was hugely enjoyable and bought Clear Skies as well), and perfect review scores didn't convince me to buy Halo.

Finally, strong reviews convinced me to buy Sands of Time, which I now rate as my personal number 1 favorite game of all time, but you couldn't get me to buy anything that involves rearranging gems in a grid even if you paid me to play it, never mind the review.

The one thing I don't understand in that list is publisher reputation. Publishers are so large, and publish so many games nowadays, from so many different teams, that I don't think publisher reputation is meaningful in any way.

michael_sylvain's picture

Erm, Ico first time round, to name but one? Critical success often doesn't tally at all with financial success. Yes, there's a bunch of contracts where bonuses depend on Meta scores, but even so...

The bit the article misses is that reviews can create or contribute to the kind of buzz that leads to awareness of games, what genre they're in, or lead to recommendations. But the score in itself isn't anywhere near as important.

And the amount of argument that scores cause is totally out of proportion witht their relevance. That's more about certain parts of the community than ther number.

Alex_V's picture

I don't agree. Many of the biggest-selling titles of the past few years have been received coolly, especially on the Wii. Wii Play (58 on MC), Wii Fit (80), Mario Kart Wii (82), Guitar Hero 3 (82), umpteen Madden and Need For Speed titles. And many critically acclaimed games go on to sell very little. On the Wii again - Zack & Wiki, Okami, Little Kings Story.

VIB's picture

review scores are fairly pointless, but well-written reviews can be very helpful and interesting, especially if its personalised.

Simon_Winstanley's picture

Completely agree. Give me a description of the game and will be the judge of whether i think it is good or not. A reviewer is surely always going to be bias on the score based on the type of games they prefer, as no one likes all types of games.

Mooks's picture

I agree completely - I find the content of the review far more useful in deciding upon a purchase than the number at the end. However, there are those, particularly younger consumers, who with little money to spend and not as sure of their own opinions of what they like and dislike, can easily be swayed from one game to another by a difference in score.

toadwarrior's picture

Game reviews are awful that's why they don't matter. It's just something for fanboys and publishers to look at and masturbate over.

Aionic_Kid's picture

Besides, review scores are actually very little informative as most magazines tend to score from 7 to 10 and not use the whole scale. Then people need to discern the difference between a 76 and a 77... totally ridiculous. I'm in favor of broad scores, preferably 1-5. I also liked Electronic Gaming Monthly's review section back in the day; it was subjective but still quite rigorous, they gave 10s and 0s.