Surveys, the Silent Truth

The fall-out from the Warhammer Online: Age of Reckoning class cuts continues, but Mythic is not sitting idly by. They are staunchly defending their decisions, unlike so many companies that make decisions and tell the community to shut up. The part that amazes me about all of this, is that so much of the change was directed by Mythic's stellar beta feedback systems.

One of the tools being used is surveys, which allow beta testers to provide feedback immediately from inside the beta test, right after an event occurs. A system I hope will make it into launch. I've argued in the past that the feedback mechanisms in beta tests are often lackluster and generic. Most games just put up a forum, e-mail address, and a standard feedback form. All of which are susceptible to the loud minority taking over.

Not so with surveys, and this quote from Mark Jacobs, WAR's top dog, really highlights the power of a good feedback:

I am really disappointed in losing the Choppa. Choppas are great. The Choppa rocks. However, in certain terms, the class wasn’t rocking. It’s so funny, because even on our Beta boards people are saying, "I cannot believe you’re taking it out. I know the feedback wasn’t great on it, I know that it wasn’t exciting, but why are you taking it out?" Well, that’s the whole point! We’re taking it out because we have gotten this kind of feedback, because we know that people aren’t playing them.

This happens all the time on boards. You get a small percentage of the population acting like they speak for the entire population. However, this time, we’ve got the data. We know just how many people were playing the Choppa or the Hammerer. We know just how long they took to level. We have the surveys. We have all this data going in. There was one post on the forums that said, "I know I have provided a lot of hard feedback on the class…" Well, that's the whole point!

Anyone arguing that Mythic is haphazardly, at the last minute, changing key game mechanics is wrong. Mythic is measured and consistent with their changes; with copious amounts of data to back up their decisions publicly. The only lacking aspect has been the rather slow release of information, which should pretty much be done now that Guild Beta is in full swing.

I truly love WAR that much more every time I read interviews from the development team. It is refreshing to see a development team that has been around a while, actually using the knowledge that has been gained through a dozen failed projects, along with the data gained from great in-house feedback mechanisms.