Thank god Im getting a degree and approaching the hardware side. It has its issues, as well, but yuck on testing.

I did some game testing back in December. The game was an FPS and it wasn't really fun. We did multiplayer matches for an hour a day which was alright.. But apart from that we were walking around maps bug hunting.
The frame rate was all over the place as well which got quite nauseating after a few hours. Still, we had massive COD4 LAN matches during lunch :D
Now I'm testing mobile phones instead and I have a lot more job variety now thank God.

The article pretty closely mirrors my own experience working as a tester on Perfect Dark Zero at Microsoft's Samammish campus. One thing I'd like to add is that the atmosphere was really a lot of fun. It was four guys to a cubicle and you'd have another team right behind you so there was constant chatter about whatever anyone wanted to talk about. That was a big part of what made it bearable once we started working 65+ hour weeks to get the game ready to ship. One thing that's funny about the fatigue is that it destroys the quality of your work because you're a lot less likely to notice any problems, but it doesn't stop you from rocking at the game. There were nights when I'd be completely out of it by 10 or 11 (which means I was totally useless for noticing or reporting any bugs) but I'd still be pretty high on the kill board. (these were balance and load tests, so we were supposed to be playing the game rather than checking for holes in geometry or missing textures or anything like that)

Maaan I had a completely different idea for gaming test. I thought that the designers (not the whole team off course) stayed with testers for some time to see their reaction. Also I imagine that there were a psicological team to hear reports of how they feel after playing 60 hours of games some games, like FPS or GTA stuff.

TheKbob:
Thank god Im getting a degree and approaching the hardware side. It has its issues, as well, but yuck on testing.

If you think that hardware doesn't have its own testing, better think again :P.

Thinking about it, besides the obvious reasons like increased pay, I wonder why coding experience isn't required of all games' testers. If all these guys are doing is pointing out the bugs/problems in code, things may be expedited in the design process if the testers, knowing something about code, could provide better details on a bug than "You can wall clip on level 5". Heck, even being able to fix the simpler bugs in the code they're testing could give the game designers more time to tackle the tougher bugs/balance, or to implement more features...or a normal work week.

Just my 2 cents.

Yeah! And game testers would cost more. Easier to enslave the game designer.

Necrohydra:
Thinking about it, besides the obvious reasons like increased pay, I wonder why coding experience isn't required of all games' testers...

That is a surprisingly complicated question and the answer will necessarily vary from publisher to publisher. Part of it is money, part of it is politics, part of it is logistics.

In general, there are two types of testers: Black box and White box. You're talking about white box testers (testers that actually break in to the code, do more granular debugging and actually fix 'easy' issues.) Black box testers test from a player's perspective and very few ever get deeper than poking through .ini's or game logs for specific information. Most testers in the games industry are strictly black box. There's a few good reasons for that and I'll try to cover a few of them.

As noted in the article, Microsoft does look to promote testers with coding experience. But as you mention, cost is a major concern for making this a requirement. According to Game Developer Magazine's 2007 Salary Survey, Quality Assurance Testers with less than 3 years of experience earn an average of $39,063 (which feels a little high, honestly, and is definitely a piece of information I need to take to my next performance review...), while your average programmer makes somewhere in the neighborhood of $83,383. Your average Designer (the third lowest on the survey) earns around $63,649. (The true travesty in the survey is that Business and Marketing are the highest compensated persons in the industry, but that's another subject entirely.)

As mentioned in the article, publishers do hire testers that can code. However, they're primarily involved in building testing tools and automation. They're also paid a hell of a lot more than your average black box tester, so there are usually only a small handful per game team (sometimes across an entire QA Department.) Generating and debugging these tools tends to take up a HUGE amount of time for these people. So actually getting their hands dirty and black box testing the product is virtually impossible. These Test Engineers are greatly appreciated (though usually silently) by the rank-and-file. Test automation usually kills a large amount of the drudgery in testing a title. Unfortunately, some tasks can't be completely automated, so you have to pay someone to sit around and watch things. Black box tester to the rescue.

With the growing complexity of console titles and the surge in MMO production (arguably some of the most complicated software systems on the planet), there is a HUGE need to test these things at run time. The simple fact remains that you just don't know what's going to happen until you turn the key. If you can find me a coder that can produce 100% error free code, 100% of the time, I'm going to start praying to every God known to man because the end times are nigh.

One of the other issues with hiring exclusively test engineers can be in the structure of the publisher itself. If "testers" are fixing bugs, then why aren't they on the production team? It mostly comes down to money here too, but Executive egos get involved and shit gets ugly. Again, complicated and will definitely vary from publisher to publisher.

Just one last thing, my experience in QA has been mostly positive. The hours can suck, the people can be even worse, but I'd never trade my 100+hr weeks for anything. The war stories are what makes things fun. I've gotten to work on a wide variety of products and it's really pushed my ability to think critically, solve problems, manage stress, build relationships with coworkers and a whole myriad of other useful life and job skills.

Not that anybody ever really cares about testers, but I like talking about this sort of stuff, so if people have questions, I'm more than willing to answer nonspecific stuff. ;)

Anyway, I'm going to shut up now for risk of just rewriting the article. :)

Edit: I lied about the shutting up thing. Another problem with testing is one of perception. A large segment of people (from Executives down to entry level designers) think they can test a game and that involves little more than "playing" for 8 hours a day. It's really unfortunate that the perception still exists, especially considering there is such a vast difference between play testers and QA testers. While play (balance, flow, progression) testing is part of the QA experience, the job extends to things far beyond them and the sad thing is that in my experience developers (Producers, Leads, Execs) are less likely to listen to game play feedback from QA testers than they are from focus groups. There are reasons behind that, I'm sure. But when somebody that is familiar with every in and out of your product says something is screwy with an aspect of "subjective" game play, I would think the decision makers and the product owners would listen...

I appreciate testers, greenlaser :) I am a designer in a game company, and I remember the months that the game went in full-testing all too well. It's surprising how many bugs a level can have. Testing a game can be a real bore, so i can appreciate how people manage to play a level for the hundredth time and still are sharp enough to find a new bug.

Testing is more then 'just play around a bit'. You need to know what you are looking for, and how to find it.

I think I missed a couple of "sweeping generalizations" in my post, but I appreciate the tester love. :)

Thinking of a certain game, testers didn't even check if players would enhance all their powers with Damage Boosters. Having devious, leet-speaking, spawn-camping, gold farmers is the way to go.

The_root_of_all_evil:
Thinking of a certain game, testers didn't even check if players would enhance all their powers with Damage Boosters. Having devious, leet-speaking, spawn-camping, gold farmers is the way to go.

I have to wonder; did you work on said game? If not, why assume that it was testers that missed something. 9 times out of 10, this isn't the case and the decision to fix or not fix is handed down from way above the testing level. For major things or even the minor little gripes people have with games, to fix or not to fix is just not something testers have control over. As a matter of fact, it's really not even as simple as pointing a finger at a single position in the development cycle and saying "omgz. ur fault!"

PEWPEWGreenLaser:

I have to wonder; did you work on said game? If not, why assume that it was testers that missed something. 9 times out of 10, this isn't the case and the decision to fix or not fix is handed down from way above the testing level. For major things or even the minor little gripes people have with games, to fix or not to fix is just not something testers have control over. As a matter of fact, it's really not even as simple as pointing a finger at a single position in the development cycle and saying "omgz. ur fault!"

Well, I did Beta test it. I also mailed them the comment that 'Perhaps it might be a good idea to test it'. I watched as they had to bring in a massive patch halfway through Alpha to fix that very problem and I watched them blame the testers.

So...I'm putting my bets on 'quite sure' especially as the idea had been on the forums on Beta, and six months into Alpha.

It's not the only huge design flaw I spotted; and I'm really not looking that hard. If you have a game that requires you to do X to Y, and then you give players chance to boost X or boost V; you have to be thinking they'll want to boost X.

To those think that QA testers "play games all day" consider this analogy. You can QA test a piano by hitting every single key, listening to the note it makes, and writing down which notes are off and by how much(You also have to include detailed instructions on how to hit that key, preferably with attached photographs and a video of you hitting the key). If you think that that is "playing a piano" then you are right, QA testing is "playing games."

Granted I only have one day of testing experience, it was a 14 hour day. It sounds to me like the experience at Nintendo and Microsoft are understandably ideal compared to other places. Working for a third party company, you can be expected to pull 50-70 hour weeks frequently, and with a 6 month contracted that won't be renewed unless you put in every hour that's asked of you. Not taking the overtime can be considered to be taking vacation time. You're in a dark room for 10-16 hour days with temperatures that are either freezing or boiling hot depending on when the cooling systems kick in, and replaying the same level of a game 15-20 times a day. As somebody who could happily play video games for entertainment every day until I die, I'd say doing QA/Testing is poison for being able to enjoy games. There's too much of a good thing, and then there's too much of a not-so-good, unpolished and stressful thing.

dgrassa:
Granted I only have one day of testing experience, it was a 14 hour day. It sounds to me like the experience at Nintendo and Microsoft are understandably ideal compared to other places. Working for a third party company, you can be expected to pull 50-70 hour weeks frequently, and with a 6 month contracted that won't be renewed unless you put in every hour that's asked of you. Not taking the overtime can be considered to be taking vacation time. You're in a dark room for 10-16 hour days with temperatures that are either freezing or boiling hot depending on when the cooling systems kick in, and replaying the same level of a game 15-20 times a day. As somebody who could happily play video games for entertainment every day until I die, I'd say doing QA/Testing is poison for being able to enjoy games. There's too much of a good thing, and then there's too much of a not-so-good, unpolished and stressful thing.

Spot on. Only people that have really tested video games know what it's like. Microsoft and Nintendo may be the goal for all testing labs to shoot for, but the reality of the situation is that most places just aren't sunshine and rainbows all day.

As far as the too much of a good thing goes, you reach a saturation point with testing where you're able to compartmentalize "playing" for pay and playing for fun. It took me about a year of hellish work schedules (almost 6 months straight of 12hr/day overtime, with over 450hrs clocked one January...) to reach that point. I'm sort of a workaholic though.

The (at the time) Director of my department and my manager took one look at me on my 29th straight day of 16's and said "Go home. You look dead." I really did, but I was a contract employee and sort of terrified that it was the beginning of the end of my tenure with that company. There's also something a little disheartening about being given "special" treatment when the rest of your team is still in the shit, so to speak. It sucked terribly at the time, but it's honestly one of my favorite war stories. :)

I remember after a month or so of testing I started being ultra critical of the games I was playing on my own time. I also remember being at a party and having this conversation:

Stranger: So what do you guys do?
Me: I test video games.
Friend: I'm a paramedic.
Stranger(turning to me):That's cool. What's it like?
Me: He's the one that saves lives!

its a nightmare testing a game that's not fun at all, especially when it doesn't get any more fun with following iterations!

I was going to comment on the article, but first...

Eagle Est1986:
Very informative, I always assumed that each developer had their own testers though, it never occured to me that Microsoft would have their own testers for games, as they don't develope their own games.

What? Have you never heard of Microsoft Flight Simulator? Mech Commander? Allegiance? Age of Empires? Halo? Those are all made by studios wholly owned by Microsoft (or which were wholly owned at the time of release). Microsoft Games Studio owns several studios, and publishes for many more. Which leads into my comments on the article, which focuses on one small part of testing...

There's at least three sets of testers for most games -- Developer, Publisher, and in the case of console games, Certification. This sounds mostly like Publisher testing, which is primarily black box for very political reasons (mainly because over-worked developers can snap when some little tester they don't know starts telling them how to do their job, whether they're right or not).

Developer-end testing is primarily white-box, and their job is initially to simply test the code (particularly fixes) just after they get put in. Once full-testing gets up, they end up primarily reproducing bugs found by black-box testers so the developer in question get get additional information about it, or see the issue in action (please note: developers, on average, do not play their own game often, and some never do, which is something that has eternally baffled me). In some cases, these guys will also fix quick typo and scripting bugs, but often they aren't allowed to (more political reasons, generally, though also some practical ones of keeping the number of people with direct access to the code down, and to ensure those changes are put in by someone who will know if they might have side-effects).

Publisher-end testing is almost always entirely black-box -- and in some cases, the tester is specifically ordered to play dumb (e.g. "even if you think this issue is related to another, don't act like it, write both up separately, and don't even refer to the other one in your write-up"). Some of this is directly practical (if the issues aren't actually related, confusion ensues if they get written up together), but mostly it's maintaining the black-box mind-set of not assuming anything about the underlying structure of the code.

Certification Testing, from what I know of it, is almost entirely about checking for very technical aspects of the game -- things like ensuring common terminology (e.g. calling it the "Left Analog Stick" and not "Left Thumb Stick" in the controller options menu), button prompts (nothing annoys a console maker more than seeing the buttons that look like those on the rival console), and trademarked words are used properly (e.g. PlayStation® always has a capital P and S, and if in an all-capitals font must still have a larger P and S than the other letters), and so on. They also do a quick shakedown to ensure the game doesn't crash regularly or have any really glaring functionality problems (they don't care if it's fun, just if it's playable).

And as a final comment to root_of_all_evil -- as the low man on the totem pole, testers are almost universally blamed even when they can directly point to the issue they wrote up on something. Alternately, the testers did test it and since it fit the design document they were given ("Boosters can be applied to powers? Check. All powers can be boosted? Check. Any limits on boosting the the official design? Nope? Check. Ok, next test suite."), didn't write it up because it was by design (despite any problems it may have seemingly caused), and were told not to question the almighty game and balance designers who clearly knew more, even if they never played their own game.

I have never seen a game shipped that didn't have some known issues. I have also never seen a game ship without several bugs entered regarding play balance, exploits, and design flaws which were waived by the developers and producers as either by-design / intentional, marginal / not worth the time to fix, or "too difficult to do" which end up being in every review as a bad mark, and in the case of multiplayer games wreak all sorts of havoc. I have never seen a review note something we missed, and I can't think of any cases where a major exploit showed up that we didn't catch first. My bet is on there being someone between the devs and the testers that caused the issue you described in said game, and not it being simply missed by the testers entirely unless it only had very limited in-house pre-alpha testing.

Just out of interest, how can one get a job as a game tester?

shMerker:
The article pretty closely mirrors my own experience working as a tester on Perfect Dark Zero at Microsoft's Samammish campus. One thing I'd like to add is that the atmosphere was really a lot of fun. It was four guys to a cubicle and you'd have another team right behind you so there was constant chatter about whatever anyone wanted to talk about. That was a big part of what made it bearable once we started working 65+ hour weeks to get the game ready to ship. One thing that's funny about the fatigue is that it destroys the quality of your work because you're a lot less likely to notice any problems, but it doesn't stop you from rocking at the game. There were nights when I'd be completely out of it by 10 or 11 (which means I was totally useless for noticing or reporting any bugs) but I'd still be pretty high on the kill board. (these were balance and load tests, so we were supposed to be playing the game rather than checking for holes in geometry or missing textures or anything like that)

Which means you were paid more than you were worth for the company.

sanspec:
Just out of interest, how can one get a job as a game tester?

It depends on what you are interested in doing as a tester. As noted in the article there are numerous different disciplines and types of test available. Localization, black box, white box, usability, automation, etc.

If you have zero professional and/or coding experience, your best bet is getting in touch with a technical contracting company and having them hook up an interview for you. Games are becoming far too expensive to produce generally to warrant hiring entry-level test full time. Obviously location is important as well, I live in Seattle so that was not a problem for me personally.

Great article by the way Alan, I'm still dreamy eyed. =P

And still after all that button mashing: Button, button, who's got the button?

Interesting read. It's nice to see an article thats actually got a bit of depth. Most things on the escapist are just wise ass stabs at the industry :P Not that there is anything wrong with that haha. In any case I always imagined guys sitting around eating pizza and playing games, turning every now and then to give a big thumbs up or thumbs down to their own reflection in a one way mirror with a bunch of scientist types taking notes. You certainly opened my eyes hehe.

Thanks again, good job.