Remote Mobile User Testing Workshop (UXDC 2017)
-
Upload
userworks -
Category
Technology
-
view
25 -
download
1
Transcript of Remote Mobile User Testing Workshop (UXDC 2017)
Workshop:Remote Mobile User TestingNew Tools for Moderated Mobile Testing at a DistanceTristan Wilson • Dana Douglas
www.userworks.com@userworks
Workshop: Remote Mobile Testing2
Research ScenarioRemote mobile story time
Workshop: Remote Mobile Testing 3
Research Scenario
Email from a Client
Workshop: Remote Mobile Testing 4
Research Scenario
Let's break it down. What stands out?
Hey Tristan,We have a new mobile app that the team is working on that we’d like you to test. It’s a functional prototype, so users should be able to click around and see some of the functionality. We want to test it with a geographic mix of smart phone users in the U.S. – let’s say 8 participants. We’d like to learn whether people can navigate and understand the labels, as well as just get their general impressions of the app.Our design team in San Francisco would like to observe the sessions live – is that ok?As always, the budget’s pretty tight, so let’s keep this as inexpensive as possible.Can you work up a plan to present to the team next week?Thanks,Dana
Workshop: Remote Mobile Testing 5
Research Scenario
Major takeaways from email• Testing for a new mobile app• Functional prototype• Want to test user interaction• 8 participants desired • Need a geographic mix of participants• Want to observe sessions remotely• Budget is "pretty tight"
What will our study look like?
Workshop: Remote Mobile Testing6
Remote TestingA Bit of Background
Workshop: Remote Mobile Testing 7
What We Mean When We Say...
Remote TestingParticipant and researcher in different geographical locationsSynchronous research, live-moderated
Mobile Modern, portable “smart” devices Most commonly smartphones and tablets
Workshop: Remote Mobile Testing 8
Benefits of Remote Testing?
More naturalistic environment
Lower cost & increased efficiency
Greater convenience
Expanded recruitment capability
Workshop: Remote Mobile Testing 9
Drawbacks of Remote Testing?
Potential reduction in quality of data
Expanded spectrum of technical issues
Reduced scope of research
Diminished participant-researcher interaction
Workshop: Remote Mobile Testing 10
Drawbacks Specific to Mobile?
Range of OS versions across devices
Lack of mobile interaction tracking/capturing Touch, gestures, etc.
Participant currently has to share their own screen Creates the potential for privacy concerns
Wireless can lead to reduced speed & reliability for data transfer
Workshop: Remote Mobile Testing11
Existing MethodsYesterday's Remote Mobile
Workshop: Remote Mobile Testing 12
The Browser ResizeConnect to researcher’s computer remotelyInteract with browser sized to mobile dimensionsUse laptop/desktop PC, – not mobile device
° Researcher resizes browser on their own computer to match dimensions/pixel density mobile device
° Testing conducted like traditional remote session on PC, but mimicking mobile
° Participant interacts with the mobile-sized browser remotely with mouse/keyboard
Workshop: Remote Mobile Testing 13
The Standalone Webcam
° Participant uses an external webcam to take video of themself as they interact with mobile device
° Video feed is live-streamed to researchers located remotely
Not an accurate depiction
Position external webcam to capture deviceStream interactions to remote researchers
Workshop: Remote Mobile Testing 14
The “Laptop Hug”Wrap arms around reverse-facing laptopInteract with device in view of laptop webcam
° Like standalone webcam method, participants stream a video feed of their mobile interactions
° Instead of a separate webcam, participants use their laptop (or tablet) to take video
° Hinged laptop facilitates camera angle; easier on participant
Workshop: Remote Mobile Testing15
New OptionsTomorrow's Remote Mobile
Workshop: Remote Mobile Testing 16
Ingredients for New Remote Mobile
Workshop: Remote Mobile Testing 17
Mobile Mirroring
Commonly via native wireless “casting” protocols:
Google Cast(Android)
AirPlay(iOS)
Workshop: Remote Mobile Testing 18
Mobile Streaming
Directly streaming from the mobile device itself- or -
Re-capturing & streaming a locally-mirrored screen using a PC
Workshop: Remote Mobile Testing 19
Setup A
Workshop: Remote Mobile Testing 20
Setup B
Workshop: Remote Mobile Testing 21
Setup C
Workshop: Remote Mobile Testing 22
Setup D
23
Remote Mobile Setup Types
PARTICIPANT RESEARCHER
Workshop: Remote Mobile Testing24
Dream ToolCharacteristics of an Ideal Solution
Workshop: Remote Mobile Testing 25
Characteristics of the Ideal Tool
Revisiting the client's email: • Testing for a new mobile app• Functional prototype• Want to test user interaction• 8 participants desired • Need a geographic mix of participants• Want to observe sessions remotely• Budget is "pretty tight"
Workshop: Remote Mobile Testing 26
Characteristics of the Ideal Tool
What characteristics would you be looking for in a software tool that can help you
meet your client’s needs?
Workshop: Remote Mobile Testing 27
Characteristics of the Ideal Tool° Ease of Use
° Painless for participants to set up and use; not intimidating° Quick and simple to initiate screen sharing from both moderator and participant
perspective° Allows for remote mobile mirroring without a local computer as an intermediary° Easy for moderator to implement and control
° Price & Affordability° The cheaper the better if all else is comparable
Workshop: Remote Mobile Testing 28
Characteristics of the Ideal Tool° Performance & Reliability
° Minimal lag time° Can run alongside other applications without impacting experience/performance° Accurate representation of participants' actions and gestures
° Range of Features° Ability to carry out other vital aspects of research in addition to screen sharing
(e.g., web conferencing, in-app communication, recording)° Platform agnostic: fully functional on all major mobile platforms, particularly iOS
and Android. ° Allows participants to make calls while mirroring screen
Workshop: Remote Mobile Testing 29
Characteristics of the Ideal Tool° Privacy & Security
° Allow participants to remotely control researcher’s mobile device via their own
OR, if participant must share their own screen:
° Considers participant privacy by clearly warning when mirroring begins° Provides participant with complete control over start and stop of screen sharing° Snoozes device notifications while sharing screen° Shares only one application rather than mirroring the whole screen
Workshop: Remote Mobile Testing30
DemonstrationMobizen
Workshop: Remote Mobile Testing 31
MobizenA relatively simple mirroring tool.
Platform: Android, iOS
Price: Free (Android Only) $3.99/mo (Enables iOS)
Features: - Mirroring- Streaming- Recording- Annotation
Workshop: Remote Mobile Testing 32
Mobizen
What setup type is Mobizen?
iOS:
Android:
Workshop: Remote Mobile Testing 33
Android Demonstration
1. Moderator installs Mobizen on computer2. Participant installs Mobizen on mobile device3. Sign in with same account on both devices4. Mod select mobile device from computer and connects5. Participant accepts connection on mobile device
Workshop: Remote Mobile Testing 34
Discussion
How would you rate Mobizen?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Workshop: Remote Mobile Testing 35
Break
Workshop: Remote Mobile Testing36
Research Logistics
Workshop: Remote Mobile Testing 37
Research Logistics
Participant Recruiting
Workshop: Remote Mobile Testing 38
Research Logistics
Participant Privacy and Security
Workshop: Remote Mobile Testing 39
Research Logistics
Recording & Observers
Workshop: Remote Mobile Testing 40
Research Logistics
Technical Setup with Participants
Workshop: Remote Mobile Testing 41
Research Logistics
Running the Session
Workshop: Remote Mobile Testing 42
Lessons We've Learned
Planning ahead is key
Practice makes perfect
Always have a backup
Put participants at ease
Tailor recruitment to make your life easier
Workshop: Remote Mobile Testing43
Group Demos
Workshop: Remote Mobile Testing 44
Group Activity° Learn about a tool with your group° Test out the tool with a mobile device° Think about this process in the context of a remote usability test° Rate the tool on the following:
° Participant Ease of Use° Moderator Convenience° Affordability° Performance & Reliability° Range of Features
° Present your tool to the rest of us
Workshop: Remote Mobile Testing 45
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Group Recap: Zoom
How did you rate Zoom?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
Workshop: Remote Mobile Testing 46
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Group Recap: Join.me
How did you rate Join.me?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
Workshop: Remote Mobile Testing 47
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Group Recap: Reflector 2
How did you rate Reflector 2?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
Workshop: Remote Mobile Testing 48
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Group Recap: AirServer
How did you rate Airserver?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
Workshop: Remote Mobile Testing 49
ParticipantEase
ModeratorEase
Price &Affordability
Performance & Reliability
Range ofFeatures
1
10
Group Recap: TeamViewer
How did you rate TeamViewer?° Participant Ease of Use° Moderator Ease of Use° Price & Affordability° Performance & Reliability° Range of Features
Workshop: Remote Mobile Testing 50
What Tool Would You Choose?Back to the email: • Testing for a new mobile app• Functional prototype• Want to test user interaction• 8 participants desired • Need a geographic mix of participants• Want to observe sessions remotely• Budget is "pretty tight"
Workshop: Remote Mobile Testing 51
What Tool Would You Choose?
What tool would you propose to use in response to your client’s request?
Workshop: Remote Mobile Testing52
The FutureWhat Comes Next
Workshop: Remote Mobile Testing 53
The Future: Remote Mobile ToolsIdeal Tool Dimensions
Ease of UsePrice & Affordability
Performance & ReliabilityRange of FeaturesPrivacy & Security
Workshop: Remote Mobile Testing 54
The Future: Remote Mobile ToolsSpecific areas for improvementFeatures
° Ability to work reliably with Android AND iOS
Privacy° Screen sharing of one particular app rather than the whole screen° Sharing control of the researcher’s mobile device screen
55Workshop: Remote Mobile Testing
Tristan WilsonUser Experience Specialist
[email protected]@tristanwilson
Dana DouglasUser Experience Specialist
Workshop: Remote Mobile Testing 56
Appendix: Workshop Handouts
Workshop: Remote Mobile Testing 57
Appendix: Workshop Handouts
Workshop: Remote Mobile Testing 58
Appendix: Workshop Handouts
Workshop: Remote Mobile Testing 59
Appendix: Workshop Handouts