Make Your Games Play, Teach, and Socialize Better: Usability & Playability Techniques From the Field

Jason Schklar

Game & User Experience Consultant

Make Your Games Play, Teach,

and Socialize Better:

Usability and Playability

Techniques from the Field

Triangle Game Conference April 7, 2010

What I know a lot about

Entertainment

Games

Human

Computer

Interaction

Attitudes &

Behavior

April 7, 2010Triangle Game Conference

What I know a bunch about

Social

Computing

Social Games


What I came here to learn about

Serious Games


Back to what I know a lot about

Most of them are gamers.

They already play great

games that are usable,

playable, and social.

For Serious Games to be

effective they need to be

great games.

I’ve worked on lots of games

where players make complex

decisions in real time.

I’ve worked on many UIs and

tutorials that engage, teach,

and train players quickly and

effectively.

I know a lot about Serious

Game players

I know a lot about teaching

complexity through game play


I also know a lot about Game Improvement

One way to make your game better is to identify where

users experience pain and transfer that pain back onto

the shoulders of your development team.

Ideally this is an iterative process that involves testing

with users.


Game Improvement Via Pain Transference

Embarrassment & Worry

[Image of participant/s

struggling]

Angst & Fear

[Image of collaborative

approach]

Step 1. Measure the pain by

studying your users.

Step 2: Discuss causes and

solutions.



Resentment

Step 3. Make fixes. Lots and

lots of fixes.



Resentment Relief / Euphoria

Step 3. Make fixes. Lots and

lots of fixes.

Step 4: Lather, rinse, repeat.

Validate fixes, find new issues.


Testing with Users: Why Do It?


Discovery

Trial

Conversion

Evangelism

Each link in the user experience chain represents a

potential falling off point in the virtuous cycle.

The way to assess user pain is through user testing.

User-Testing: Why NOT Do It?

There are lots of barriers to testing with users efficiently

and effectively...

Sometimes they’re legitimate reasons

Sometimes they’re excuses


It costs too much…

We can’t afford dedicated facilities, fancy equipment


It doesn’t integrate with our process

We can’t wait weeks for a long report on outdated code

We don’t have time to incorporate feedback


It dilutes/distorts/corrupts our vision

We don’t design by focus group


We already do it, thank you very much

We bring in friends and family when we can, have our

admins and HR folks play the game, run focus groups


Testing with Users: Techniques from the field

Techniques I’ve used with great results

Techniques that address and overcome barriers


Case #1: RITE Usability

PC Real Time Strategy

Developers were

working too fast for

reports to be relevant

I worried that I wasn’t

providing the best

suggestions

Doesn’t fit their process

May distort their vision

The Challenge The Barriers



Created a remote dev studio within the usability lab

Discussed issues and implemented fixes in real time



Pros:

Integrated very well with the team’s development process

Because we validated content and features as they went in, late

additions and changes to the game were made with confidence

Great results in terms of critical review and commercial success

Cons:

High $$$ cost (team travel)

High energy cost (team travel)

Some loss of efficiency (there’s no place like home)


Case #2: Discount Extended Playtesting

Console RPG

Weeks to going gold

Unsure whether it was

too hard or too easy

Currently testing mostly

with existing fans

No dedicated facilities

Costs too much

Doesn’t fit our process

Already do it




Used spare equipment

Recruited non-fanboys

Operated in real time



Pros:

We found several progress blocking issues – and fixed them

We validated core tutorials and progression with non-fan boys

The process took days (hours) not weeks

Called up on a Friday afternoon; we were done by Wed

Cons:

We didn’t get a very nuanced view of the user experience

This wasn’t a “polish” study; it was an “attrition reduction” study

We didn’t get a chance to validate our fixes


Case #3: Raise the Acceptance Bar

Several small widget

development teams

Not enough time and

money to test them

Some projects were

highly confidential

(internal testing only)

Varying fidelity of mocks

Costs too much


Distorts our vision




Weekly opportunistic tests with interns, low-fi mocks

Less “acceptance” testing and more “rejection” testing



Pros:

Fit within existing process (weekly sprint based design reviews)

We caught LOTS of issues while at the pencil and paper stage

Cheap

Cons:

Our test users were still more savvy than the typical customer

You can only learn so much testing with non-interactive mocks


Case #4: Improving Existing Processes

Peripheral-based game

Needs “broad appeal”

Publisher didn’t have

resources, so developer

was doing it themselves


Distorts our vision

Already do it




Offloaded user-testing work from devs/designers.

Created a makeshift lab using existing offices and tech

Different audiences to find convergence and divergence



Pros:

Able to have the team observe how players of all backgrounds

approached, failed, and succeeded at the game

Improved their process and results using a combination of their

own space and equipment (and some elbow grease)

Cons:

Still not entirely hands free (although it moved the load from

design and development to production and IT)

Some offices/studios can’t spare even two rooms


Summary: Techniques > Barriers

Usability & Playability Techniques

Excuses

1. RITE

Usability

2. Discount

Extended

Playtest

3. Raise the

Acceptance

Bar

4. Improve

Existing

Process

Costs too

much

Doesn’t fit our

process

Distorts our

vision

We already do

some of it


Testing with Users: General Principles


Getting the most out of testing with users

User experience lead has some form of independence

backed by key stakeholders and project leaders

Collaboration yields group therapy, not group think

Think about and discuss fixes in real time as you

discover issues (because no one reads the full reports

or goes back and watches the videos)

Feed fixes back into the game as quickly as possible

Test again

Oh, yeah...

Keep your mouth closed and your eyes and ears open

Empathize with the users to come up with meaningful

stories that explain their (often) puzzling behavior

Make sure you address underlying user experience

issues, which are not always what users complain about

Give players what they need, not what they ask for

Implications for Serious Games


Applying findings to Serious Games

Magnitude of errors: What is a “Sev 1 Issue”?

Loss of life

International incident

Higher drop out rate

Cost savings still apply

Reduce training time (people may even train on their own time)

Reduce CSS costs and overhead

Broaden your candidate pool cheaply

Harness the power of social and community

People already talk about stuff: Track it and surface it

Integrate into existing social and community work flows


Questions & Answers

My Contact Info:

e: [email protected]

t: @jackalshorns

w: www.InitialExperience.com

Resources

IGDA – Games User Research SIG

LI Group: http://www.linkedin.com/groups?gid=1873014

IGDA Page: http://www.igda.org/user-research

SIG Resources: http://sites.google.com/site/gamesuserresearch/

Microsoft Game Studios User Research:

http://mgsuserresearch.com/default.htm


mailto:[email protected]

http://twitter.com/jackalshorns

http://twitter.com/jackalshorns

http://www.initialexperience.com/

http://www.linkedin.com/groups?gid=1873014

http://www.igda.org/user-research



http://sites.google.com/site/gamesuserresearch/

http://mgsuserresearch.com/default.htm

Make Your Games Play, Teach, and Socialize Better: Usability & Playability Techniques From the Field

Technology

Transcript of Make Your Games Play, Teach, and Socialize Better: Usability & Playability Techniques From the Field