Usability Testing in Video Games - Motion Controls b

Usabi l ity Testing for Vi deo Ga mes: Moti on Controls

by Aloke Datta-Chaudhuri

2Usabi l i ty Testing for Video Games: Motion Con trols 3

Word Count - 3279

Contents

Introduction p1

Background p1

Research and Recommendations p5

Conclusion p11

References p12

23 3Aloke Datta-Chaud huri k11 60647

I ntroduction

This report is a critical review of current practice with in the field of User Experience Design. The report will focus on the practice of usability testing in video games, specifically motion control games. The report will look at different methods and aspects of usability testing currently practiced or advised with in the video games industry, how they differ from traditional usability testing methods including those recommended in CIF and then suggest guidelines and recommendations for how to usability test motion control games by adapting or incorporating the methods discussed.

Background

Why is Usability Testing Becoming More Common in the Games Industry?

Usability testing of video games is a field of user research that has rapidly increased in the past 10 years. The video games industry has grown over recent years reaching a bigger and more varied audience and with it the need for usability testing in video games. Video games are saturating our lives much more as new devices such as smart phones and tablets a long with consoles and com-puters implement gaming. With this growth comes greater opportunity for new game genres but also the need for excellence to be the market leader in that genre, always improving the experience on sequels so not to lose your audience to competitors. It is this level of competition which is ever increasing that has generated greater attention to details such as user experience design and us-ability testing of games. Developers also recognise the need to quickly understand how usability can be achieved on new platforms and hardware, with the biggest variation of games devices then ever before which means making a multi-platform game usable is increasingly difficult.

Three versions of PES 2011, the iphone, XBOX 360 and Wii respectively, all with differing interactions and inter-faces which brings with it a range of usability issues for what is essentialy the same game.


Competition Breeds Innovation and with it Unknown Usability Territory

“For some time, we have believed the game industry is ready for disruption. Not just from Nintendo, but from all game developers. It is what we all need to expand our audience. It is what we all need to expand our imaginations.” Satoru Iwata - President of Nintendo, 2006

In 2006 Microsoft released the XBOX 360 to continue to grow its share in the core gamers market along with Sony who were developing the even more powerful Playstation 3. The intense competition had already resulted in a poor sales performance and reviews of Nintendo’s Game Cube which lead the company to drastically re-think it’s strategy and innovate a new games console hardware to at-tract a different type of gamer and gaming experience. The result of this innovation was the Nintendo Wii, the first console whose primary control mechanism was a motion controller. Nintendo marketed the console and its hardware heavily to a wider audience highlighting the design of their motion con-trollers offering ‘an intuitive, natural way to play games.’ (Nintendo.co.uk, 2011) as well a s being a healthier way to play games. This attracted a wider age range of customers propelling the console to out sell its competitors where they were previously the weakest sellers with the Game Cube:

figures taken from Wikipedia, Jan 2012

The response from Microsoft and Sony was to develop their own version of motion control technology for their existing consoles in order to take a share of the market that Nintendo had opened up. The Playstation introduced the ‘Move’ in late 2010, a controller that closely resembled Nintedos Wii-Mote. Microsoft’s response was the release of ‘Kinect’ which advanced from the motion controls of the Wii by removing the need of input from a controller instead using a camera that tracks your movement and recognises gestures. The Kinect, upon release went on to become “the fastest selling consumer electronics device ever” (Guinnessworldrecords.com. Retrieved March 10, 2011)

Despite the clear push for motion controlled systems at this moment in time, the games have not received positive reviews from critics or users to match the sales of the hardware with classically con-trolled games still being the more popular choice of games. This has left many frustrated with the lack of creativity and effectiveness in games to match the innovation of the hardware.

‘so far there have been a handful of qualified successes and many more whiffs -- games that don’t control as advertised or that only replace button commands with finicky gestures.’ IGN, November 2011

Wii - 89.36 million units sold Xbox 360 - 57.6 million Units Sold PS3 - 56 million Units Sold


Nintendo wanted and succeeded in creating a games revolution however to make these new controls intuitive to users, games and designers have to adapt to the hardware and understand exactly how users want and expect to use motion controls. Up untill now the focus has been on innovating motion control hardware with less emphasis on how games designers should interperate the technology.

‘Game control mechanisms have tended to evolve gradually over time However, the movement-sensing functions of the Wiimote do not have any precedents to help designers understand which make most intuitive sense to users for different actions.’ Ben Weedon, 2007

Such drastic change can create excitement but it also creates expectations which leaves uninformed designers guessing as to what users want without having the luxury of analysing previous iterations and patterns. The role of user research and usability testing is to help understand the expectations of users and help develop a pattern of usability for motion controls that adds to the overall gaming experience.

Research and Recommendations

‘As of yet, no one size fits all easy approach exists to incorporate user research into game develop-ment. Rather, there are a range of tactics and tools that may be appropriate depending upon the project and the resources at hand’ Schaffer, 2009

To develop a better understanding of how to make motion control video games more usable there needs to be clearer guidelines for usability testing. As motion control gaming is a relatively new tech-nology such guidelines do not exist however there are companies that recommend certain techniques and conduct research on how to approach usability testing for video games in general however there are varying opinions on how much video games can be incorporated in to usability testing theory and must be judged carefully:

‘while some standard usability testing procedures and doctrine, such as those used with productivity software, are applicable to video game design, many are not. In fact, a large portion of productivity software usability ideals are in direct contrast with their use in video game design’ Timothy Ballew of 8-bit Bear Consulting, 2008

‘there’s nothing about video games that changes the fundamental nature of usability’ Tim Fidgeon of Spotless Interactive 2011

To evaluate the best approach for usability testing of motion control games we must look at the key aspects of the standard usability testing including those recommended in CIF along with video games industry research and opinions and draw the best conclusions to apply to motion control gaming.

To give the report greater focus the research and recommendations will focus on sports sim-ulation motion control games as these have shown to use the most significant motion input.


Test Objectives

‘This section describes all of the objectives for the test and any areas of specific interest.’ CIF Tem-plate, 2003

‘Video game usability testing sessions are most useful when the factors of effectiveness, efficiency and satisfaction are carefully considered and applied to the video game context in a meaningful way.’ Tim Fidgeon, 2011

With any type of product that under-goes usability testing there are certain objectives of the test that are unique to the product, its genre and concept. With in the context of motion control games and sports simulation one of the main objectives would likely be that the controls and gestures should be effective and easy to execute with the actions providing a more realistic, immersive and therefore satisfying game. It is important to note that efficiency effectiveness in a gaming sense does not relate to the ease of completing game tasks or objectives as there must be a sense of challenge:

‘Take, for example, ease of use. This is a basic design principle and is adhered to (or should be) vehemently in productivity software. In terms of video games (specifically game play), a game that is easy to use will be uninteresting.’ Timothy Ballew,2008

The objective of motion controls being easy to use and efficient is highlighted greater then with clas-sically controlled games as there is greater complexity and range to the input of motion controls then pressing buttons however the amount of motion control input needs to be monitored so not to make the experience tedious or overly complex.

‘there may be a tendency amongst developers to try and show off what the Wii-mote can do without considering what really works for the end user.’ Ben Weedon, 2009

(above) Wii sports will have different test objectives to Grand Slam Tennis in particular a lower complexity in the

techniques for using the motion controls.


Participants

‘This section describes the users who participated in the test in terms of demographics, professional experience, computing experience and special needs.... If there are any known differences between the participant sample and the user population, they should be noted here’ CIF Template, 2003

General data should be recorded as stated in CIF of age, gender, computer experience, education and product experience but there are also other specific factors that must be recorded for motion control sport simulations.

Most sports simulations are competitive games with a multi-player aspect, if the game is multi-player, especially local multi-player it is impossible to test the game fully without recruiting participants in groups. Whether the game is focused mostly on group play or only aspects of it are group play choos-ing participants and testing should be done by considering how participants will interact with each other competitively and socially and if necessary, individual testing can be done separately for the sin-gle player aspects. Recruiting participants for group testing is an unconventional method in usability testing as most software and websites are designed for an individual however group testing has been done within in game usability testing and the recommended procedure is to select participants that are familiar with each other. Familiarity between participants will allow them relax and play the game socially how they would in a ‘real life’ situation with friends where they feel free to be competitive and express emotions.

‘Players seem to be more relaxed when in a group of people that they know. This makes it more likely that they will behave as they do naturally. Players grouped with strangers are generally more reserved.’ Schaffer, 2009

Another aspect that should be considered when selecting participants for motion control games and more specifically motion control sport simulations is the fitness level of the participant. Motion controls are naturally more strenuous and active for the user. Games that require prolonged movement in its game-play will have different effects on players with different levels of fitness thus creating a possible bias in the test results if unrecorded, this is an issue that is not raised in classically controlled video games or typical software/web design testing.

Participants Environment

‘The section should include all the detail required to replicate and validate the test. It should include appropriate configuration detail on the participant’s computer, including hardware model, operating system versions, and any required libraries or settings.’ CIF, 2003


The participants environment of video games has significant differences to the environment of more conventional testing of web sites and software as the focus of the activities are almost opposite. Video games consoles are most often set up in relaxing social spaces where as software and web sites are most commonly used in a work environment and more personal spaces. There are further differences with motion control games where having a well lit room and removing noise interference is less important then having space to move and the room temperature at a reasonable level not to make prolonged movement uncomfortable

Many companies that usability test video games have suggested that conducting the test with in the participants home environment should be considered however the ideal environment would be in more controlled conditions in a usability lab that reflects a home environment such as a living room particularly for group testing. The important factor is to try and create an environment that the partici-pant would consider a natural gaming room

‘We would recommend that video games are tested using various different set-ups, rather than solely in a usability testing facility. We would strongly recommend that any video game usability testing facil-ity should be flexible enough to mimic a participant’s own preferred gaming set-up.’ Tim Fidgeon of Spotless Interactive 2011

The most likely key differences to a conventional lab set up would be a couch and television with plenty of room to move as supposed to a desk and office chair with a monitor.

Equipment

CIF recommends that audio and video recording devices should be stated if used which is common practice in conventional usability testing however there should be a greater level of detail for record-ing the movement of the participant in sports simulation motion games as it is this input that will be put under greater scrutiny. With the increased movement expected with motion control games it is recommended that multiple cameras be used to collect all data and movement, this would include recording the screen, the participants and if it is a group test, the social interaction and emotional responses.

For a sports simulation motion game tests should be encouraged to use equipment to record in greater detail aspects related to fitness, technique and reactions. This would require more special-ized equipment but would result in data that can be better analysed. Examples of such equipment includes:

Heart Monitors - Monitoring the heart rate can help indicate the difficulty level of sports simulation games and the level of fitness required to complete goals in the game, helping designers to tweak them to suit their target audience. Body temperature and breathing rate can also help monitor this all of which have been developed by EmSense who specialize in bio-metric usability testing.

‘EmSense has created cutting edge biosensor technology that measures a player’s physical re-sponse to game experiences. They have a headset that measures brain-waves and other sensors that measure temperature, breath, heart rate, physical motion, and eye movement.’ Chris Swan, 2009


Motion Capture Suits - An example of how this can be used in a test can be seen in a case study by ‘Playable Games’ who used motion capture suits to help identify the differences in movement and posture when comparing the use of the old Wii-remote with the newer more sensi-tive and accurate Wii motion plus controller

The test went on to describe how analysisng the data from the motion capture suits highlighted

the improvement in user interaction with the game when using the new controller with participants re-flecting more realistic movements for the actions being played.

‘The motion capture data revealed that players who were motivated to achieve used a more realistic style of play when using the Wii MotionPlus (as supposed to the Wii-mote)’ Gemma Boguslaws, 2009

Usability Metrics

Possibly the biggest difference between conventional usability testing and video game usability testing is the choice of metrics. The focus in games is always severely leaning towards getting good satisfac-tion ratings as the main purpose of games is to have fun where as a web site and piece of software usually tries to focus on productivity and ease of use which is why measuring effectiveness and effi-ciency is so important.

However with motion control games and sports simulation there are elements of effectiveness and ef-ficiency that can be measured to greater detail then classicly controlled games. Errors can be counted for the number of times that a participant tries to carry out an action that does not register correctly or it ithe participant executes the wrong gesture expecting it to be the right one, these issues are relatively common occurrences as described by the following video games user testers:

‘For example, what is the best way to allow users to open a door? Wiggle the Nunchuck or press a but-ton?’ Ben Weedon, 2007, ‘Designing games for the Wiimote’

Talking about Kinect’s ‘Sport Season 2’ - ‘Issues did start to appear if more than one person attempted to interact with the menu at any one time. The system seemed to struggle, switching control back and forth between the players’ Alistair Gray, 2011

the use of motion capture suits would help analyse these errors as well as possibly highlight different errors where the wrong gesture was resulting in the right action.

Another significant difference between conventional user tests and video game testing is the construc-tion of tasks or rather the lack of tasks, making it impossible for games to be measured for time on task or task completion. The reason for this again is down to the philosophy of games being about having fun and not focusing on being productive.

Wii-mote and Motion Plus results from a motion capture suit iused by ‘Palyab Games’


The set up of many usability tests for games revolves around ‘playtests’ rather then tasks, where par-ticipants play the game for a set amount of time with out any instruction to complete specific objectives. This allows for better satisfaction feedback and user behaviours to be observed however also gives an indication of effectiveness by measuring the number of errors incurred during this time. The main purpose of playtesting as supposed to task testing is that it reflects a more natural way of playing the game and assigning tasks to be observed would have significant implications on satisfaction rating of he game.

Microsoft, the pioneers of playtests recommend that playtests observations for participants should be ‘typically 1 hour’ long (Denni Wixon, Microsoft, 2009) however this should be considerably less for sports simulation motion games as they can involve strenuous activity and would cause frustration and possibly harm to participants if played for this long. Instead the game should be ‘playtested’ with breaks in between matches (using tennis/football as an example) or rotated round a group of participants giv-ing time for individuals to recuperate.

Satisfaction is the key metric for video game usability testing and can be measured in a similar way to conventional testing, with questionnaires, user comments and observing emotions. One of the biggest difficulties that is not such a major problem in software and web site testing is participant narration. This often gives valuable insight in to the users thought process, level of frustration and explanation as to where the product is going wrong. In traditional usability testing the moderator often asks the par-ticipant to narrate each step they take in the process and even asks questions during tasks so that the participant can elaborate on certain decisions. This method is not considered a useful one with in video games testing as much of game play is down to short reaction times, particularly for sports simulation games that involve movement of the body.

‘In the traditional usability test the instructor interrupts the participant every now and then to ask ques-tions. In this study it was found that this is not always plausible in games. This is because the interrup-tions cause unnecessary difficulties to the participants. One potential way to avoid this problem is to have a mixture of think aloud and uninterrupted play’ Sauli Laitine, 2006

it’s hard for many people to play a game well and talk about it at the same time. Often they focus on one or the other, with less than ideal results.’ Adams Greenwood-Ericksen, Eric Preisz, Shawn Staf-ford, 2010

Experienced user testers have recommended that participants should not be prompted to talk but instead adopt a more natural approach and let them comment when they feel like. Some have recom-mended testing in pairs or groups specifically to get a dialogue that can be recorded and analysed which would still be more natural and less intrusive then moderator questions.

‘in this study, the participants tested and played the games in pairs. Their comments during the game-play were used to collect feedback on potential unclear elements in the game interface, issues with the game controls, and more generally feedback on the enjoyability of the game’ Henriette (Jettie) C.M. Hoonhout, 2009

The best time for the moderator to ask direct questions is once the participant has finished the test.


Conclusion

My research in to usability testing for motion control video games has highlighted key differences when compared to traditional usability testing. The key source of these differences stems from the focus of games being about fun and entertainment however there is scope for more traditional methods of test-ing to be applied to the motion control actions themselves in order to identify errors of input and rec-ognition of gestures. The important factor in testing for motion control games is that you get a range of data that can be analysed along side each other to draw conclusions from. This ideally includes adopt-ing the use of more high-tec equipment such as motion capture suits and heart monitors as with inno-vative and complex gaming hardware there should be equally innovative techniques to analyse them.

The recommendations outlined in the report are based on existing research however as motion control gaming technology is in its infancy it will take time to build a pattern of usability that can be applied to the many genres of gaming and these recommendations will change with each game that gets tested. During my research it became obvious that even with in the genre of motion control gaming there were multiple sub-genres each with there own unique styles to which a general guideline would not always help, this highlights the complexity of usability testing in games.

With Nintendo continuing the Wii motion controller in to its next console generation, Microsoft releasing Kinect for Windows and Playstation planning to develop games for the Move over the next 5 years, we could start to see a greater understanding of how motion controls should be applied to gaming. Usabili-ty testing has a big role to play in this process however it has to adapt to the needs of the video gaming industry and the innovation of motion controls as the concepts will undoubtedly start to find its way into web site and software navigation, the so called traditional subjects of usability testing.

12Usabi l i ty Testing for Video Games: Motion Con trols

References

Noah Schaffer, Katherine Ibister, 2009, Chapter 1, Game Usability, Advancing the Player Experience

Satoru Iwata - President of Nintendo, 2006, nintendolab.com

Michael Thomsen, 2011, IGN, Opinion: Skyward Sword is Motion Control’s Future

Ben Weedon, 2007, Develop, Designing games for the Wiimote

Timothy Ballew, 2008, 8-bit Bear Consulting, Usability Testing and Video Games: Designing for Fun

Tim Fidgeon, 2011, Spotless Interactive, Video game usability testing – factors to consider

Chris Swan, 2009, Chapter 9, Game Usability, Advancing the Player Experience

Gemma Boguslaws, 2009, How do better motion controllers affect the game experience?, Playable Games

Alistair Gray, 2011, An expert review: the user experience of Kinect games

Sauli Laitine, 2006, Do usability expert evaluation and test provide novel and useful data for game development?, Journal of Usability Studies.

Adams Greenwood-Ericksen, Eric Preisz, Shawn Stafford, gamasutra, 2010,Usability Breakthroughs: Four Techniques To Improve Your Game

Usability Testing in Video Games - Motion Controls b

Documents

Transcript of Usability Testing in Video Games - Motion Controls b