Remote Research, The Talk.

Thursday, July 1, 2010

flickr/thetruthabout


REMOTE RESEARCH

Definition:

Research where the participant is not sharing physical space with the researcher.

In person (95%)Remote (5%)

% of qualitative research done remotely*

*Source: BP’s best guess


#UXREMOTE


WHEN DOES REMOTE WIN?


Marv.Creative director UX designer product manager developer IA.


Marv’s Boss.He’s always on a plane and loves birds.


Hey Marv, I want to save some money on research.


NATIVE ENVIRONMENTCHEAP ETHNOGRAPHY, NOT CHEAP LAB.


TIME-AWARERESEARCH IS THE BEST


I’LL TRADE YOU TIME FOR PHYSICAL PRESENCE


INVOLVED OBSERVERS

Easier to collaborate in real time.


TEXT

Geography


WHEN DOES REMOTE LOSE?


Hey Marv, I want feedback on our MMO video game from 13yo English speaking kids in France & Latin America with webcams.


I guess, Boss.

Might be hard to do with remote testing.


BANDWIDTH

Yes, it’s an issue.


TRANSLATION

It’s expensive and di!cult to do live.


MINORS

Consent is a bear.Especially if they’re under 13.


FACES

Most users don’t have webcams.Webcams are a little invasive.


SUPER DUPER SECURITY


AN EPIC FAIL.


What Happens When You Let Go Of A

Balloon?

How To Find Actual English Speakers:


WHAT WENT WRONGCoins were way more valuable than we expectedPeople mis-assessed their English levelsBandwidth was really a problemNobody plays Habbo when their parents are home to consent


DESIGNINGREMOTE!STUDIES


Yo Marv, get me some insight on WHY people are bailing out of checkout.


Sure, Boss.

Let’s intercept & observe 10 users live with screen-sharing, and do 10 self-moderated with usertesting.com


TIME-AWARE TASKSTold to find purple pumps.

Doesn’t care.

Has to wear this for sister’s wedding this weekend.

Whole di!erent interview.


REAL TIME OBSERVERS


TEXT


PROTOTYPES, WIREFRAMES, SKETCHES?


HECK YES.


10 YEARS244 STUDIES2,915 PARTICIPANTS (MODERATED)95 ORGANIZATIONS19,610 HOURS


60% PROTOTYPE!RESEARCH


A DINOSAUR FAMILY EXPLAINS INFORMATION ARCHITECTURE


Marv-o: are their any UX web apps we can use for online research?


Sure, boss. There is an AVALANCHE of UX TOOLS.


“At least a little behavioral”

Surveys. Web Analytics.


FIVE CATEGORIES

• Moderated (screen-sharing + audio)• Self-moderated (usertesting.com)• Automated (loop11, usabilla)• Fancy Analytics on Steroids

(crazyegg)• Diary (revelation)


Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live


Tasks On Concepts

For A Long Time

People Talk To A Machine

Talk To People

Fancy-Ass Analytics

Task Bar On Images

Task Bar On A Live Site


I.A. & Navigation

For Ad Agencies

Ammunition

Insight

Just Better Analytics

Still More Ammunition

More Ammunition


Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

Deep Behavioral UX

Insight


Opinions

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live


Ad Agencies Love It

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live


“Online Qualitative Research”

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live


Jared Spool Hates It

Conceptual

For A Long Time

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live


THAT WAS 293 WORDS


USABILLA

Clicks & QuestionsHeatmapsYou do your own recruiting$49 - $199


TEXT

LOOP11 (UN-MODERATED)


TEXT


* $49/mo* Integrated VOIP * iPad Viewing


TEXT


Conceptual

Self-Moderated

Moderated

Web Analytics

Automated Static

Automated Live

Automated Static


WHAT WE USE.

A LOT.

• Ethnio• GoToMeeting + iShowYouHD• Usabilla + Usertesting.com


RECRUITINGFOR REMOTE


So Marv, how do we find people for these studies?


Stop harassing me and get o! that weird plane.


RECRUITING OPTIONS

Grab someone in the hallwayGrab someone who isn’t directly involved in your projectIM your friends for a quick lookPanels and recruiting agenciesReal customers from marketing listsSite intercepts

Realness


GOLD STANDARDLive recruitingGet as close as you can


RECRUITING OPTIONS

EthnioLinks or house ads to online formsRecruiting agenciesCustomer email listsPanelsCraigslist/online ads


USE ETHNIO WHEN

Website accessSome budgetQuotas are relatively simple


USE FORMS WHEN

You can place a link but not codeLow budgetQuotas are relatively complexOptions:

Google DocsWufoo


USE AGENCIES WHEN

No website accessPeople whine about first-time visitorsHigher budgetOnline recruiting failed


CUSTOMER LISTS

Sub-segmentsOwnership requirementsNo website access


PANELSThursday, July 1, 2010

SORT!OFThursday, July 1, 2010

CRAIGSLIST

When other methods don’t workNo doughTakes a whileEmbarrassing to admit to your friends


LIVE RECRUITING MATH

Target: 6 recruits/hourAllow for: 1% response (low average

You Need: 10,000 uniques/day


TEXT


Paying Incentives

Collect minimal personal infoDeliver quicklyAppropriate for location


HOW!TO!BEgraceful

When All Your

TECHNOLOGYBREAKS


WHAT THEY WORRY ABOUT


WHO ARE YOU AND CAN I TRUST YOU?


IF YOU CAN SEE MY SCREEN,

CAN YOU SEE MY FACE?


WILL YOU BE ABLE TO TAKE

CONTROL OF MY COMPUTER?


WILL YOU HELP ME GET THE PLUG-IN

OFF MY COMPUTER AT THE END?


WILL THIS WORK IF I HAVE AFIREWALL/PROXY/STRICT IT

DEPARTMENT?


IS THIS GOING TO END UP ON

YOUTUBE?


OXYMORON?REMOTE EMPATHY


EMPATHY TIP

Get someone on the phoneGive them an impossible puzzle


HYBRID STUDIES

Numbers actually do lie sometimesAnd they don’t tell you the why3-5 remote interviews is all you need


MOBILE REMOTE RESEARCH?


?


ADVANCED TECHNIQUESOne to many moderation

Multi-threadingThursday, July 1, 2010

PORTABLE RESEARCH


BURN THE SCRIPTFIND OUT WHAT MATTERS.


ADVANCED TECHNIQUESHigh Emotions


TEXT


THE FUTURE


remoteusability.com


@boltron


Remote Research, The Talk.

Technology

Transcript of Remote Research, The Talk.