Survey Research using Emerging Technologies - m-s-g.com · 2015-08-06 ·...

1Trent D. Buskirk, Ph.D. 6/10/15JOS 30th Anniversary Conference Short Course: Survey Research Using Emerging Technologies

Survey Research using Emerging Technologies:

Trent D. Buskirk, Ph.D.JOS Conference June 10 and 11, 2015

AAPOR Short Course, 2014

Considerations for Design, Data Collection, Sampling

and Recruitment for Smartphone and Tablet

Based Surveys


Instructor: Trent D. Buskirk, Ph.D.

Current Position: Vice President of Statistics and Methodology @ Marketing Systems GroupLives in: Saint Louis, MOFocus on:

Use of new technologies for data collectionMobile Phone Sampling Dual Frame Designs and WeightingMobile Analytics and Paradata Collection and Analysis

Addicted to:Playing Prince to my little Princesses APPS and iPhonesSAS, SUDAAN and RTennisBoost 101.9 (www.boost1019.com)

http://www.m-s-g.com/Web/Index.aspx

http://www.boost1019.com/

http://www.m-s-g.com/Web/Index.aspx


Colleagues at MSG for their assistance and support and for providing thumb drives for participants

My Family for their patience as I persisted through course prep

JOS Conference Organizing Committee for the Invitation to Present

3

Acknowledgements and Thanks!


Agenda & Goals

Background: Nuts and Bolts of our new Mobile UniverseUnderstanding the landscape that is before us to set the stage New types of data available – burden, privacy and availability

Framework for Data Collection on Mobile DevicesActive versus Passive data collection methods

Developing Emerging Best Practices: A blend of old and new

Types of Mobile Surveys (and what we know now from the Literature)Free Resources for Mobile Web SurveysRecommendations for Internet/App based survey solutions (Top 14 EBPs)

Day

1


Agenda & Goals


Recommendations for Internet/App based survey solutions (Top 14 EBPs)

Computing – just how do you make these surveys? From Websites to Mobile Web Surveys (Web design methods for Mobile)Overview of “Mobile Versions” for Popular Online Survey Packages

Goal: Leave with a solid appreciation of:Potential role of new technologies in data collection;Different ways you can use smartphones to collect survey dataSome practical “best practices” for deploying mobile surveys

Day

2


Reference Note

I have included web links as well as QR codes for nearly every data source we will discuss today.

Some popular QR code reader APPS for your smartphone include:

I-nigmaKaywa ReaderBeeTagg

The following website allows you to enter your exact mobile provider and provides a recommendation as to the QR code that is optimized for your phone’s camera

http://www.mobile-barcodes.com/qr-code-software/

http://www.mobile-barcodes.com/qr-code-software/


Section 2:

Emerging

Technology and

the Rise of

Smarter Cell

Phones

6:59


What are Feature Phones?

A feature phone is a mobile phone that combines the functions of a PDA and a mobile phone. These models serve as portable media players, web browsers and camera phones.

Feature phones generally refer to devices that are more low-end compared to higher-end Smartphones.

The most important difference between a feature phone and a Smartphone is the kind of operating system installed in it.

http://en.wikipedia.org/wiki/File:Nokia_6020.jpg

http://en.wikipedia.org/wiki/File:Nokia_6020.jpg


What is a Smartphone?

A Smartphone is a high-end mobile phone that combines the functions of a personal digital assistant (PDA) and a mobile phone.

In general, a Smartphone will be based on an operating system that allows it to run applications, use e-mail, and surf the internet.

Apple's iPhone runs the iOSBlackBerry Smartphones run the Black -Berry OS Other devices run Google's Android OS, HP's webOS, and Microsoft's Windows Phone.


For the Visual Learners in the Room – a Summary

The Picture Summary!

http://www.about.com/



Smart vs. Feature Phones Worldwide

Source:http://www.nielsen.com/us/en/newswire/2013/how-the-mobile-consumer-connects-around-the-globe.html


Smartphone penetration for US Mobile Subscribers

over the past two years

“To every thing there is a season, and a time to every purpose under the heaven,” Ecclesiastes 3:1


What is a Tablet Computer?

A tablet personal computer (tablet PC) is a portable personal computer equipped with a touchscreen as a primary input device, and running a modified desktop OS designed to be operated and owned by an individual.

Tablet PCs are larger than smartphones. Common form factor dimensions are (height, width, thickness and weight):

5-inch Tablets: 6" x 3.2" x .4" @ .75 lbs.7-inch Tablets: 7.5" x 5" x .5" @ 1 lbs.9-inch Tablets: 9.3" x 6" x .5" @ 1.2 lbs.10-inch Tablets: 9.8" x 7" x .5" @ 1.4 lbs.13-inch Tablets: 12.5" x 8" x .5" @ 2 lbs.


Devices come in all sizes (and shapes)

Screen dimensions are measured in pixels (px) both for height and width. Another measure that combines these to give an overall measure is dots per inch (dpi)1 megapixel = 1,000,000 pixels.Common widths (in pixels) for various devices:

320 px

768 px

1024 px

1680 px




Mobile Devices introduced in 2013

Source: Millennial Media, 2013: http://www.millennialmedia.com/mobile-intelligence/mobile-mix/


Mobile Devices Covering the USA

240 Million Mobile Users (age 13+) in the U.S.

91% of All U.S. Adults have a Cell Phone

156 Million Smartphone Owners in the U.S.

(age 13+)

64% of U.S. Adults have a Smartphone

82 Million Tablet Owners in the U.S.

42% of U.S. Adults have a Tablet

Sources: comScore (2015) and Pew Research (2015) http://www.comscore.com/Insights/Presentations-and-Whitepapers/2015/2015-US-Digital-Future-in-Focus

10% of American Adults are Mobile-Only

Internet Users; 15-21% among 18-29 YOs

http://www.comscore.com/Insights/Presentations-and-Whitepapers/2015/2015-US-Digital-Future-in-Focus


Mobile Devices Covering the World

Sources: http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15

4.77 Billion People Worldwide Using

Mobile Phones

2.03 Billion People Worldwide Using

SmartPhones

400 Million People Worldwide Own

Tablet Devices

788 million mobile-only internet users

by 2015 worldwide (Cisco, 2014)


Smartphone Users Worldwide: 2012 – 2018

Source: emarketer (2014): http://bit.ly/1uWDLtg

http://www.emarketer.com/Article/Smartphone-Users-Worldwide-Will-Total-175-Billion-2014/1010536

http://www.techvibes.com/blog/tablet-adoption-rate-exceeding-that-of-smartphones-2013-10-15


Mobile device penetration continues to rise in the U.S.

Source: Comscore, 2014 http://bit.ly/1hh9j8L


Tablet Penetration in the EU-5 (2012-2018)

0.0%

10.0%

20.0%

30.0%

40.0%

50.0%

60.0%

70.0%

2012 2013 2014 2015 2016 2017 2018

Shar

e of

pop

ulat

ion

UK Spain Germany France Italy

Source: http://www.statista.com/statistics/271001/penetration-rate-of-tablets/

http://www.statista.com/statistics/271001/penetration-rate-of-tablets/


Smartphones in the U.S. - They’re almost everywhere!

Just over three in four (77%) mobile subscribers in the U.S. owned a smartphone during the most recent three-month period (Oct-Dec 2014), Nielsen, 2015

Android Operating System most popular smartphone OS in US followed by Apple IOS.

Source: Nielsen (2015):http://bit.ly/1MDPB4j


Most Common Smartphone Operating

System Per Country

Note: These estimates are based on Mobile Web Activity rather than Device Ownership

Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/

http://www.nielsen.com/us/en/newswire/2013/mobile-majority--u-s--smartphone-ownership-tops-60-.html

http://bit.ly/1MDPB4j


A different picture for Tablets

SOURCE: MILLENNIAL MEDIA’S Q2 2013 MOBILE MIX™ REPORThttp://www.millennialmedia.com/mobile-intelligence/mobile-mix/


Most Common Tablet Operating System

Per Country

Note: These estimates are based on Mobile Web Activity rather than Device Ownership

Source: StatCounter: GlobalStats (2015) http://gs.statcounter.com/

http://www.millennialmedia.com/mobile-intelligence/mobile-mix/


Smartphone Penetration in the US by Key

Demographic Features

Source: Nielsen (2015): http://bit.ly/1MDPB4j

Source: Nielsen (2014):http://bit.ly/1pyhyfG


Age and Income Level Strongly Associated

with Smartphone Ownership

Source: Nielsen Wire (2012) : http://www.nielsen.com/us/en/newswire/2012/survey-new-u-s-smartphone-growth-by-age-and-income.html

2012

Source: Pew (2013) http://www.pewinternet.org/2013/06/05/smartphone-ownership-2013/

2013

http://bit.ly/1MDPB4j

http://bit.ly/1pyhyfG

http://www.returnpath.com/wp-content/uploads/resource/email-mostly-mobile/Return-Path-Email-Mostly-Mobile1.jpg

http://www.nielsen.com/us/en/newswire/2012/survey-new-u-s-smartphone-growth-by-age-and-income.html

http://www.pewinternet.org/2013/06/05/smartphone-ownership-2013/


Slightly Different Picture for Tablets Bigger

Screen, Smaller Income

The age distribution of Tablet users seems to skew older compared to smartphone users (comScore, 2013)

The annual HH income distribution for Smartphone users appears to skew richer than for tablet users (comScore, 2013)

Source: comScore (2013) http://bit.ly/1msbiaV


Internet traffic by device over the last 3 years (11-13)

Source: http://www.mediabehavior.com/articles/the-whole-story-internet-use-by-platform/


Mobile Devices and Internet Use Today

According to comScore’s US Digital Future in Focus report (2014):Smartphone and Mobile devices account for 57% of U.S. Internet usageSmartphones alone have exceeded Desktops http://bit.ly/1lFEDlj

According to mobiThinking (2012) approximately 25% of U.S. Mobile Web users are mobile-only.

many mobile-only are older people and many come from lower income householdshttp://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/b#mobile-only

According to Pew Research’s Home Broadband Report (2013) approximately 10% of Americans indicate that they do not have a broadband connection at home but that they do own a smartphone

http://www.pewinternet.org/2013/08/26/home-broadband-2013/


Mobile Only versus Desktop Only Internet Users (U.S.)

Source: comScore (2015): http://bit.ly/1EPWBJl

http://bit.ly/1lFEDlj

http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats/b#mobile-only

http://www.pewinternet.org/2013/08/26/home-broadband-2013/


Average daily minutes spent online by internet users worldwide as of 2nd quarter 2014, by device

258.6 267.6 256.8

74.493 108.6

0

50

100

150

200

250

300

350

400

2012 2013 2014

Dai

ly ti

me

onlin

e in

min

utes

PC, desktop or tablet Mobile

Source: GlobalWebIndex; ID 319732


Multiple Tasks for Multiple Platforms

http://www.statista.com/statistics/319732/daily-time-spent-online-device/


With the surge in Mobile Internet Use comes

the rise of unintentional mobile respondents

http://www.maritzresearch.com/ Source: Maritz Research


Mobile, Mobile Everywhere! We have to care!

With Mobile Device penetration rising along with

Increases in internet activity on mobileIncreases on the email opening activity on mobileRises in unintentional mobile survey respondents

The case is clear – we need mobile survey research strategies, best practices and resources to meet our respondents where they seem to be going! Mobile cannot be ignored!


Section 3:

Developing Best

Practices in the

New Landscape…

6:59


Thinking Differently About Technology &

Survey Data Collection

Technologies often viewed as “survey-enabling” tools, but not as “data collection vehicles” in their own rightThe advancement in apps and hardware for these mobile devices and the social networks that are accessed using them has opened the avenue for a broader spectrum of data that can be collected as part of a “survey”

Location data via captured GPS coordinates for both respondents and field workers

Scagnelli, et al. 2012 Olson and Wagner, 2015

Picture data captured via an app or via phone Scagnelli et al. 2012, Michaud, Buskirk and Saunders, 2014

Data Collection Gigs via Crowdsourcing Duan, Lai and Link, 2013


A new era of survey research ushers in new data

collection opportunities

In-person, in-the moment, round the clock data are now being collected, compared and explored in the context of surveys

Health related outcomes via apps and peripheral devices (Bluetooth) and hardware

Gregoski, et al. 2013 Abroms, et al., 2011Kumar, 2012Au-Yeung et al., 2012 (http://www.proteus.com/assets/AuYeung_NetworkedSystemForSelfManagement1.pdf)

Real-time diaries for wellness, activities and time use and trip/location dataKoskinen and Salminen, 2007 Fernee and Sonck, 2013 Graham and Cobb, 2013Lai et al., 2010Nitsche et al. , 2012


GPS analytics provided additional trip details and metrics not easily

captured via survey (Source: Nielsen, 2012)

Source: Scagnelli, Bailey et al. 2012

http://www.proteus.com/assets/AuYeung_NetworkedSystemForSelfManagement1.pdf


“Grab & Go” Purchase Locations

Each green dot is a Grab & Go store where a Nielsen panelist made a

purchase


Mobile Devices aren’t just for Respondents!

We are also using these devices ourselves to collect dataIn-person data collection via tablets is also on the rise

Seal, 2012 – http://www.burke.com/Library/Articles/John%20Seal_Article_CASRO%202012-2013%20Journal.pdf

Paudela, et al. 2013 – http://www.ghspjournal.org/content/1/2/277.full.pdf+html

https://www.dooblo.net/http://www.surveypocket.com/

Using Mobile phone–assisted personal interviewing (MCAPI) van Heereen et al., 2014 van Heereen et al., 2013

– http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3713928/

http://www.burke.com/Library/Articles/John Seal_Article_CASRO 2012-2013 Journal.pdf

http://www.ghspjournal.org/content/1/2/277.full.pdf+html

https://www.dooblo.net/

http://www.surveypocket.com/

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3713928/


Interface & Flow

Source: Bosnjak et al, 2010 http://ssc.sagepub.com/content/28/3/350.short

User interface: usually an important consideration in data collection, with new technologies it is essentialNew & extended human-computer interaction concepts:

Visual appealIntuitive flow (more than simply “logical”)Ease of use /functions are readily apparentFew, if any, text instructionsOptimize mode capabilities and reduce user anxiety and frustration

Study of why people participate in mobile surveys (Bosnjak et al 2010):

Perceived enjoyment; attitudes toward participation; self-expressiveness, & trust factors most important

Blend current understanding of CAI systems with expanding literature on human-computer interactions (use & interaction)


Thinking about Mode Choice on Multiple Devices

The emergence of devices with internet access has created unprecedented choices for online survey completion:

computer (lap/desk top), Tablet, Smartphone, FeaturePhone (for the brave)Other device (Sony PSP, your Smart TV, your Fridge, etc.)

http://ssc.sagepub.com/content/28/3/350.short


Thinking about Mode Choice on Multiple Devices

Buskirk, Walton and Wells (2013) developed a study to investigate factors related to survey mode choice

Study employed a subset of smartphone, tablet and computer users from a nationally representative probability online survey panel and replicated this study for a large opt-in panel of similar users Study preference data analyzed using Conjoint Analysis.Factors included: Survey Mode (Smartphone, Tablet and Computer as well as Paper and Pencil), Time/Incentive (5 to 60 minutes; 2 to 25 dollars), survey sponsor, Incentive Type (Cash, Card, eCode, Charity) and Survey Topic). Among the combination of survey time, incentive amt and type and mode, Tablet and Computer mode had the top 5 combinations while smartphone only had one combination preferred in the top 25 (ranked 21). The degree of preference for smartphones increased as prior exposure with survey modes (other than computer) increased. Similarly, preference for computer mode decreased as prior survey exposure to on alternate modes increased.


Thinking Differently About Technology &

Survey Data Collection and Respondent Burden

Technologies often viewed as “survey-enabling” tools, but not as “data collection vehicles” in their own right

What we normally ask ourselves:How can we use technology to conduct a traditional survey?

We should be asking:What are the questions / data elements we need to answer/collect to provide insights into phenomena of interest?How can technology be utilized to acquire that understanding / those data?What does respondent burden look like in the new landscape?Are there new data available that we can collect passively using new technologies that haven’t been easily collected before?


Respondent Burden

The degree to which a respondent perceives participation in a task as difficult, time consuming, or emotionally stressful.

Typically measured in terms of lengthor administration time of a survey.

Most common approach to countering respondent burden: incentives

Focus on “traditionally defined” Respondent Burden is no longer enough …

“The concept of respondent burden is like that of the weather: everyone talks about it, but no one does anything about it.”

Norman Bradburn, 1978

Source: Norman Bradburnhttp://www.amstat.org/sections/srms/proceedings/papers/1978_007.pdf


Respondent Burden in the Mobile World

Respondent burden in mobile world is similar to respondent burden in the regular world with a few differencesTwo Groups: Personal and Technoogical

Personal: SafetyRespondents aren’t just responding at home anymore!Steeh, Buskirk and Callegaro (2007) report respondents completing cell phone surveys while boarding a helicopter.

Personal: PrivacyNew data types like photo/voice may mean information is collected from more than the consenting respondentAge of assent on internet is younger than consent via phone (13 vs 18/19)

Personal: TimeTo complete survey tasks To load survey pages

http://www.amstat.org/sections/srms/proceedings/papers/1978_007.pdf


Respondent Burden in the New Mobile World, Cont.

TechnologicalData consumption

Both receiving and sending information (survey questions/content and submitting responses)Nielsen (2012, Q3/Q4) Estimates that 96% of mobile phone users have a data plan and that the average monthly bill is $66 (all mobile) and $93 for smartphone users.

Bandwidth Slower bandwidth increases time required to send and receive dataHigher data consumption may lead to throttling by the provider –meaning reduced bandwidth in future

Battery DrainPersistent GPS capture drains mobile device batteriesExtra HTTP requsts can drain mobile device power


Internet Access Method varies by Mobile Device Type

Internet access for US smartphone users seems to be predominantly through Mobile Connections rather than wi-ficonnections;

However this pattern is different across operating system

Source: comScore (2013) http://bit.ly/1msbiaV


Smartphone Use Statistics by Country

https://think.withgoogle.com/mobileplanet/en/

Source: Our Mobile Planet, Google, 2013: http://bit.ly/1HSHFwO


Data Plan Limits and Smartphone Users

Source: Pew Research Center (2015) http://pewrsr.ch/1FlmRbv

https://think.withgoogle.com/mobileplanet/en/


Respondent Expectations

Being involved in a scientific survey is a relatively rare occurrenceLittle, if any, expectations by respondents

Respondent experience with new technologies is much differentRespondents are more savvyTechnology changes rapidly – some respondents will find some tasks difficult or not easy to use.

[Example: Michaud, Buskirk and Saunders, 2014 Voice Data Entry]

Developing expectations from these experiencesEase of use/intuitiveSpeed Usefulness/utility or fun/entertainment or bothAbility to share experiences w/ othersLocation awarenessAuto detection/passive collection


The new paradigm: MOBILE Surveys

Smartphone Survey Design Research

Social Science Research

Buskirk and Andrus (2012)Link and Buskirk (2012)

Kinesis White Paper (2010)Buskirk et al. (2011)

Callegaro (2010)Callegaro and Macer (2011)

Peytchev and Hill (2010)Zahariev et al. (2009)

Smith (2011)Bosnjak, Metzger, Graf (2010)

Lai et al. (2010)Couper (2010)

Millar and Dillman (2012)

Human Computer Interaction

Parush and Yuviler-Gavish(2004)

Jones et al. (1999)Sweeney and Crestani

(2006)Fitts (1954)

Goldberg, Fariden and Alterovitz (2007)Zichermann and

Cunningham (2011)

Computer Science and Web

DevelopmentKoch, P-P. (2010)

Campaign Monitor (2012)Kriss (2013)

Medallia (2013)EmailVision (2013)

Html5rocks.com (2013)Pelletier (2013)

Return Path (2013)


On Emerging Best Practices in the New Landscape…

Traditional Components of Best Practice: Length/administration time of a surveyRespondent burden (time, number of phone minutes, number of surveys)Follow-up approaches/frequency, survey field periodsIncentives – cash or other tokens (gift cards, minutes, etc.)

New Components of Best Practices:Balancing user engagement with user expectations/experienceEase of use / intuitive Speed of tool / interface / appRespondent privacy expectationsRespondents sharing their experiences…Visual Appeal, user interaction with tool and gamification


Section 4: Types of Mobile

Browser Surveys

and What we

Know so Far

about Mode

Effects

16:59


Various Approaches for Survey Data Collection

via a Smartphone

Use the regular phone mode

Use Short Text Messages (SMS)Flash polls, Invitations, Synchronous Survey Interviews

App-based/administered surveys

Online surveys accessed via Smartphone BrowserPassive and Active versions

Hybrid Approach: App-Like Smartphone Browser Surveys

Source: Buskirk and Andrus (2012) www.surveypractice.org.


SMS Surveys via Mobile Devices (Phones)

SMS Surveys involve a series of short communications between surveyor and respondent (i.e. text messages)

Surveyor sends a question with answer choices to the respondent (short snipets of usually 140 characters or less)

Respondent sends back the appropriate answer

Process continues as a series of asynchronous communications

Some 79% of cell phone owners say they use text messaging on their cells (Pew, 2012)

http://www.surveypractice.org/


SMS Surveys via Mobile Devices (Phones), Cont.

Nielsen estimates that on average, 764.2 text messages were sent/received per month among all U.S. mobile post-paid/contract users.

http://bit.ly/1QkUUFY

Studies using SMS for survey data collection are emerging, for example:

Conrad et al. (2013) http://bit.ly/1gNVomsYan et al. (2013) http://bit.ly/QAL5LVMavletova and Couper (2014) http://bit.ly/1RGbEdkSchober et al. (2014) http://bit.ly/1M5sqiO


Online Surveys via Mobile Web Browser

Passive: Surveys intended to be completed online are sometimes completed via mobile phone web browser (passive mobile web browser approach).

Active: The online web-based survey is optimized for the mobile environment. (think mobile versions of survey websites)

Many survey software programs (e.g. SNAP, Qualtrics, etc.) have mobile publishing options that optimize an otherwise online survey for the mobile browser.

If you know the type of devices on which those sampled will complete the survey, you can optimize the survey design for the particular browser/screen resolution (e.g. Safari / iPhone).

http://bit.ly/1QkUUFY

http://bit.ly/1gNVoms

http://bit.ly/QAL5LV

http://bit.ly/1RGbEdk

http://bit.ly/1M5sqiO


Passive versus Active Smartphone Surveys

Passive Mobile Survey from Hotwire

Survey Appearance after a “double tap” Active Mobile Survey

Both viewed on an iPhone 5


The Literature on Passive Mobile Browser Survey

Mode Effects is now ACTIVE!

Passive mobile browser surveys are surveys developed and intended to be completed via computer.

Buskirk & Andrus (2012) hypothesized that passive mobile browser surveys could have negative survey consequences. (http://bit.ly/1f0Qvez)

Fast forward 2 years: rise of mobile unintentionals + mass use of passive mobile browser surveys = empirical findings…

Lower overall completion rates/higher break-off rates for Passive Mobile Surveys accessed on smartphones (Decipher, 2013; Wells et al., 2013; Poggio et al. 2013)Overall break-off rates among tablet users appear consistent with computer users (Decipher, 2013; Wells et al., 2013; Poggio et al., 2013)Overall median/mean completion time longer for smartphone users compared to computer users (Wells et al., 2013 [~2 minutes longer; Decipher, 2013 [~3 minutes longer], Cunningham et al., 2013 [about ¼ minute longer]).Respondents on Smartphones tend to be younger, female and non-Caucasian and primarily internet through phone users (Peterson et al., 2013; Wells et al. 2013)Faster Time to Hit Survey Start Page (Cunningham et al. 2013 [~170 min sooner]; Peterson, 2012 [~1-5 hours sooner (median)]

http://bit.ly/1f0Qvez


The Perfect Storm Goes Mobile: The intersection of non-mobile

surveys and the rise of the unintentional mobile respondent

http://www.maritzresearch.com/Source: Maritz Research

Second Quarter 2013 Device Analysis

% of Total Starts

% of Total Completes

% of Mobile Completes

Completion Percentage

Completion Time Relative to Desktop

All mobile phones 14.0% 12.4% 100.0% 75.0% 132.4% iPhone 9.3% 8.4% 67.6% 72.1% 129.7% Android 4.3% 3.7% 29.8% 72.1% 137.3% Blackberry 5.0, 6.0 & 7.0 0.2% 0.2% 1.2% 67.4% 184.4% Other mobile phones 0.2% 0.2% 1.3% 68.2% 133.1%

iPad 7.2% 7.6% NA 88.4% 99.7%

Desktop 78.8% 80.0% N/A 85.7% 7.7 minutes

Second Quarter 2013 Device Analysis

% of Total Starts

% of Total Completes

% of Mobile Completes

Completion Percentage

Completion Time Relative to Desktop

All mobile phones 14.0% 12.4% 100.0% 75.0% 132.4% iPhone 9.3% 8.4% 67.6% 72.1% 129.7% Android 4.3% 3.7% 29.8% 72.1% 137.3% Blackberry 5.0, 6.0 & 7.0 0.2% 0.2% 1.2% 67.4% 184.4% Other mobile phones 0.2% 0.2% 1.3% 68.2% 133.1%

iPad 7.2% 7.6% NA 88.4% 99.7%

Desktop 78.8% 80.0% N/A 85.7% 7.7 minutes

A recent internal study by Maritz (2013) has corroborated earlier findings by Medallia (2012):

Completion rates for Mobile users are lower than for computer users for surveys designed for completion online via computer browsers.


Passive surveys being reported in the literature

Articles dealing with online surveys being made (passively) available to respondents are making their way into the published literature and conferences.

1. Cunningham et al. (2013) http://bit.ly/RIyKGI

2. Bosnjak et al. (2013) http://bit.ly/1hckNtB

3. Poggio et al. (2013) http://bit.ly/1icvRCV

4. Poduska & Johnson (2010) http://bit.ly/PquJVf

5. Peterson (2012) http://bit.ly/1gP5Fim

6. Peterson et al. (2013) http://bit.ly/1i5WYo0

1 2 3 4 5 6

http://www.maritzresearch.com/

http://www.maritzresearch.com/

http://bit.ly/RIyKGI

http://bit.ly/1hckNtB

http://bit.ly/1icvRCV

http://bit.ly/PquJVf

http://bit.ly/1gP5Fim

http://bit.ly/1i5WYo0


Active and Passive Surveys being Reported

in the Literature

Baker-Prewitt and Miller (2013) http://bit.ly/PqzePPExperiment Compared Active and Passive Mobile Browser Surveys to Desktops and Laptops

Survey completion time, drop-out rates, straight-lining rates were all worse for passive version compared to active mobile version

Peterson et al. (2013) http://bit.ly/1i5WYo0Experiment to compare several active mobile versions of the survey to a passive version as well as an app-based version and two versions for PCs.

Survey length (about 1 to 3 minutes longer) and break-offs (2 to 3 times) were longer/higher among passive mode compared to any of the active mobile versions and both types of mobile had longer completion times than PCsSurvey satisfaction (4 Qs) ratings significantly lower for passive compared to any active mobile or pc version. (Won’t do again rating 2 to 5 times as high for passive compared to any active mobile approach)


APP-LIKE Mobile Browser Surveys

APP-LIKE Mobile Browser Surveys (A-LMBS) represent a hybrid of the APP and the Active Mobile Browser Approaches.

A-LMBS are completed via the mobile web, but they rely on active browser refreshment using a combination of PHP Programming and Javascript to create “Active” Mobile web pages.A-LMBS can be developed to run without a persistent internet connection on multiple mobile browsers (HTML5)A-LMBS offer native, APP like functionality under the mobile browser context.

In today’s browser vernacular, app-like surveys might fall under the class of Web apps which are defined as web pages that function and appear almost app like but are accessed via browsers

Source: Buskirk and Andrus (2012) www.surveypractice.org.

http://bit.ly/PqzePP

http://bit.ly/1i5WYo0


A-LMBS Example- The Got Healthy Apps Study


A-LMBS Compared to Active Mobile Web Browser

A-LMBS Approach (iPhone)

Active Mobile

Browser Approach

(iPhone)

Active Mobile

Browser Approach (Android)

A-LMBS Approach (Android)


APP-LIKE Mobile Browser Surveys in the Literature

Buskirk and Andrus (2014) report on an experiment comparing mode effects for app-like and online computer surveys http://bit.ly/1n8JLxA

Completion rates for computer mode was significantly higher than for iPhoneItem missing rates were very low across both computer and iPhone survey modesSurvey completion times for iPhone mode was significantly less than Computer Significantly higher app ownership reported from iPhone mode vs Computer

Mavletova and Couper (2013) present the results of an experiment that randomly assigned panel members (Russia) to complete a survey about sensitive topics using an app-like mobile version or PC version. One month later, respondents were invited for a follow-up survey on the other mode. http://bit.ly/1rl1aD4

Higher nonresponse, break-off ad longer completion times for Mobile Web compared to PCMeasurement differences reported across the two modes for Alcohol consumption and income with less daily consumption and lower incomes being reported on mobile vs. PCMore surveys completed outside the home in the presence of strangers on mobile vs PCNo differential affect of satisficing by type of device uses (mobile or PC).


App-based Smartphone surveys administer and collect data via an (native) app that is installed on the sampled user’s Smartphone.

Surveys can be “pushed” to the app and then executed by the end user without the need for perpetual internet connection and can be designed to take full advantage of the phone’s capabilities like camera, voice, video, image capture, etc.

App-based/administered surveys

Data transfer for completed surveys can occur once internet connection is established.

One popular example of this approach comes from Confirmit/Techneos’ Survey On-Demand Application (SODA), see:

http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx

Video File Courtesy of Frits de Geest of Confirmit

http://bit.ly/1n8JLxA

http://bit.ly/1rl1aD4

http://www.confirmit.com/Home/Community/Data-Sheets/January-2014/Confirmit-SODA.aspx


App Attack – The rise of Mobile Device Apps

According to comScore (2013)Engagement with apps was 7 times that of engagement with web browsing, on average for iPhone owners compared to only 2 times for iPad owners.

According to Flurry Analytics (2014)Time spent on a mobile device for US consumers, on average, has risen to 2 hrs and 42 minutes per day (4 minute increase over 2013)2 hours and 19 minutes on average per day on Apps and 22 minutes per day, on average for the mobile web.

Source: Flurry (2014): http://blog.flurry.com/?month=4&year=2014Source: comScore (2013): http://bit.ly/1c9vneo


App usage among Smartphone Owners by Country



Top 15 Apps within the US Measured by % Reach

Source: comScore Mobile Metrix (2015): http://bit.ly/1ycKQM4


Considerations for an APP-Based Approach

The APP based approach offers the most options in terms of formatting, gamification, and other options for data inputAnd it offers more control per operating systemOne disadvantage is that it might be too much technology for the task at hand:

Does a one-time survey need an app for normal survey tasks? Apps must be downloaded before data collection occurs

Run the risk of “App Install Breakoff”Apps themselves count against respondent data plans/onboard memory required (download and memory)Potentially increases fielding time (development and deployment);


App-Based Survey Approaches in the Literature

Link and colleagues (2013) explored recruitment for an app-based diary study that was not part of a panel, but for a cross-sectional collection of television viewing. http://bit.ly/QCXkb4Recruitment was conducted via telephone based on dual frame RDD samples for two local markets.

Once screened for eligibility, respondents were sent app-registration details

Bailey and Wells (2012) compared mode effects between an app-based survey and an online (computer-based) version of the survey http://bit.ly/1k4dBjs

Completion rates and completion times similar across the two modesMobile app respondents typed more characters, on average, compared to computer respondents on a consumer behavior question

iPhone Rs consistently entered longer responses in less time compared to android Rs


Resources for Mobile Surveys

An overview of the survey research to date on mobile devices, capabilities, privacy and legal issues and some discussion of what’s on the horizon can be found in Chapter 1 of the: AAPOR Emerging Technologies Task Force Report

Mobile Web Best Practice Guidelines produced by the World Wide Web Consortium (W3C)

Gives very detailed suggestions on how to create web pages for mobile devices, in general. Some of the information on touch interfaces, font sizes and mobile versioning of online websites may directly pertain to survey development and practice. http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap

Survey Practice eJournalGenerally has a short publishing time from submission to online posting and contains many shorter articles highlighting experiments conducted with mobile devices.

http://bit.ly/QCXkb4

http://bit.ly/1k4dBjs

http://www.w3.org/2005/MWI/BPWG/Group/Drafts/BestPractices-2.0/ED-mobile-bp2-20101202/#bp-devcap


Resources for Mobile Surveys, Take 2

Mobile Marketing Research AssociationHas a Knowledge Center where content on mobile market research is searchable by keywords. http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects

Mobile GlossariesuSamp Mobile Glossary Contains detailed definitions for many things mobile including API, GeoFencing and many other terms relevant to mobile devices and use.http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&

The European Society for Opinion and Market Research (ESOMAR)

Guidance for Mobile Market Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf

Guidance for Mobile Survey Research Reporthttp://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf


Resources for Mobile Surveys, Take 3

Our Mobile Planet Google Site Contains data from several countries on mobile behaviors by basic demographics of the user by year (from 2011 onward) for a host of countries around the world. http://think.withgoogle.com/mobileplanet/en/

Mobi-Thinking – Website Offered by dotMobi to educate practitioners about the mobile arena

Special Section dedicated to Global Mobile Statisticshttp://mobithinking.com/mobile-marketing-tools/latest-mobile-stats

Flurry Analytics (Service and Blog)Track user activity in the app with events that are defined by the app owner. May be useful to collecting paradata for app related surveys. Also have a blog that provides regular estimates of app usage across a variety of platforms.http://blog.flurry.com/ and http://www.flurry.com

http://www.mmra-global.org/search/all.asp?c=&bst=mode+effects

http://www.usamp.com/learnmore/mobileglossary/?utm_source=RFL+Communcations&

http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Guideline-for-conducting-Mobile-Market-Research.pdf

http://www.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ESOMAR_Codes-and-Guidelines_Conducting-survey-research-via-mobile-phone.pdf

http://think.withgoogle.com/mobileplanet/en/

http://mobithinking.com/mobile-marketing-tools/latest-mobile-stats

http://blog.flurry.com/

http://www.flurry.com/


Section 5: Recruiting

Methods and

Coverage

16:59


From new devices to new sampling units!

The proliferation of devices has also provided new types of sampling units and new avenues for sampling.

Take for example universes defined by owners of a certain app (like the Yahoo App).

The sampling unit might be the App itself or alternatively the id of the downloader, stratified by Market Store (Google Play, App Store, etc.)The type or version of the app (phone or tablet) may also be a stratification variable RDD, ABS sampling frames may not provide access to the universe at adequate levels (low incidence of a given app).Perhaps selection generated from the app itself,much like push notifications, is one possible way to implement selection.


Mobile device Reach and Coverage Biases

If you deploy a solution (such as an app) using only iOS app store, then you effectively will not be covering a fairly moderate percentage of smartphone users.

iOS captures 42% of Smartphone Market in US compared to Android which captures 52% (Nielsen, 2014)

Similarly, if you design an online version of the survey with no mobile optimization, smartphone users will be present (unintentional mobile respondents) but those who receive internet only through mobile devices may be missing

approximately 25% of Mobile Web users or 10% of the U.S. adult population


Android Users versus iPhone Users

Share of Users Engaged by ActivityAnnual HH income Share by Platform

Share of Users between 18 and 24

Source: comScore, 2013: http://bit.ly/QiM43x

http://bit.ly/QiM43x


Android Owners vs iPhone Owners, Part 2

Source: Estimates Recomputed from Tables Included in Pew’s Smartphone Ownership – 2013 Update (Smith, 2013): http://bit.ly/Qnt4Rr

Demographic VariableDistribution among

iPhone Users(U.S. Adults)

Distribution amongAndroid Users (U.S. Adults)

Men 45% 51%

18-34 36% 43%

55+ 27% 20%

High School or Less 20% 32%

College Degree (or More) 54% 39%

Non-Hispanic Black 8% 19%

Non-Hispanic White 80% 70%

<30K annual HH income 15% 28%

>75K annual HH income 51% 35%


Android and iDevices and Geography!

Geographic differences may also exist among the iPhone and Android User base within the U.S.

Study conducted by Mobify in 2013Using 20,000 partner websites with over 200 million visits through 2012 93% of mobile traffic came from either an iOS or Android mobile devices (tablets and smartphones)

Source Mobify (2013): http://www.mobify.com/blog/ios-vs-android-in-2012/

The more red the state, the higher the percentage of Android mobile page views in 2012. The more blue the state, the higher the percentage of iOS views. States close to 50 / 50 will be a shade of purple. More blue=higher percentage

of views from iOS

More Red = Higher Percentage of views from Android

http://www.mobify.com/blog/ios-vs-android-in-2012/


Survey Recruiting and Mobile Devices

There are many tools/strategies that are available for facilitating survey data collection using mobile devices.

To point selected respondents to your survey you could use a QR code printed on a postcard or other type written material.

The QR code (short for Quick Response Code) is a matrix representation of bytes of information associated with your web address (or location of a survey app in one of the app markets).

GOQR: http://goqr.me/ because “QR Codes created on goQR.me are completely free of charge (commercial and print usage allowed).”Requires a QR scanner to be installed however;

Use a simplified/shortened web address (URL) from a service like bitly (https://bitly.com/)

http://surveypractice.org/index.php/SurveyPractice/article/view/63/pdf bit.ly/14etvB3


Example of a missed mobile opportunity

I recently received an invitation to complete the JD Power and Associates Vehicle Reliability and Service Survey. The main invitation came via postal mailA survey reminder came via postcard here:

http://goqr.me/

https://bitly.com/


Keeping Recruiting close to the vest

(or at least the mobile device)

An alternative to QR codes and shortened URLs could be the use of NFC (near field communication, within 4 inches) tags to transfer/receive data

In areas where internet connectivity is spotty or unreliable, field workers could collect survey data loaded to their mobile devices via NFC from central computers and then when interviewing is complete, transfer of data back to laptop or central computer could again be made with NFC – all without requiring any internet connectivity. Passive NFC tags could be embedded in survey invitation papers/postcards or on posters or via kiosks and activated using respondents’ mobile devices.

Similar to QR codes, but requires no additional apps or steps to “go online” from within the QR code reader app.


S3DS #1

Email Invitations and Web Addresses for Smartphone Surveys should be as simple and straightforward as possible. Wherever possible, email invitations should be designed responsively for easy reading and navigation on smartphone and tablets.

S3DR Tip

Recruitment for Mobile Surveys via Email

S3DR = Smarter Smartphone (and Tablet) Survey Design Recommendation

Place the key information (e.g. survey sponsor, incentive…) at the beginning of the email subject.Place the survey link as soon as you can in the email invitation, to avoid scrolling.Minimize the number of “special characters” in the survey web address.If at all possible try to avoid “aliased” web addresses (especially for mobile web version).


The Got Health Apps Study Survey Invitation iPhone

Invitation Process (Buskirk et al., 2011)

All panelists initially invited on their lap/desktops via email

iPhone panelists were asked to point their iPhone web browser to:

http://mobilehealth.slu.edu/1234567890 B


Email Opens by Platform…Mobile on the rise giving

way to unintentional mobile respondents

Source: Return Path (2013) http://www.returnpath.com/wp-content/uploads/resource/email-

mostly-mobile/Return-Path-Email-Mostly-Mobile1.jpg




Email on Smartphones and Tablets

Email and the Mobile Device Reality:52% of all cell phone owners use email on their phone (Pew, 2013)

78% of smartphone owners check email on their phone. IDC and Facebook – “Always Connected” (2013)

10% Of iPad users say Mobile is their preferred device to read email and 9% for sending it. –Perion “iPad Owners Survey” (2013)

Among mobile email users, If an email does not display correctly, BlueHornet “Consumer Views of Email Marketing” (2012)

69.7% will delete it immediately;17.7% will view it on a computer7.6% read it anyway on their device .

Men open 20% more emails on mobile, but WOMEN click 10% more often on mobile e-mail TailoredMail – “It’s time to wake up and mobilize!” (2012)


If you send a respondent and email invitation, where

is it likely read?

According to Pew Research (2013, 2011) 52% of Cell Owners use email on their phone and 54% of Tablet owners use email dailyThe 2013 US Consumer Device PreferenceReport by Movable Ink reports that:

At least 35% of Apple/Android Smartphone users spent over 15 seconds or more viewing an email.

Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013


Email Opening by Device Varies by State

StatePercentage of Emails

Opened on Smartphone

(1) Texas 57.60%

(2) Mississippi 57.27%

(3) Indiana 56.22%

StatePercentage of Emails

Opened via Desk/Laptops

(1) Maine 48.85%

(2) Vermont 47.23%

(3) Washington 47.19%

Source: https://movableink.com/downloads/us_consumer_device_preference_report_Q42013


Email Formatting Recommendations

EmailVision provides additional recommendations pertaining to optimizing email deliverabilityFor Subject lines and sender options, EmailVisionrecommends:

Rather than generic emails such as [email protected] use relevant email addresses such as:

[email protected], [email protected] [email protected]

enable the recipient to understand the intent of the email and potentially reduces auto spam classifications.

http://www.emailvision.com/sites/default/files/emailvision-deliverability-bestpractices-2011-03.pdf

http://www.emailvision.com/sites/default/files/emailvision-deliverability-bestpractices-2011-03.pdf


Email subject: Smartphone View

Android Smartphone resolution 480 by 8003.7. inches screen size

93


Responsive Email Designs for Survey Invitations

Campaign Monitor’s E-Publication “Responsive Email Designs” offers several considerations for developing mobile responsive emails including:

Single column layouts (500 – 600 pixels)

Links/Buttons minimum target area of 44 X 44 pixels

Minimum font displayed on iPhones is 13 pixels – smaller font than this will be upscaled and formatting may change

Call to action (i.e. “start survey button” or survey link) should appear toward the upper portion of the email to avoid scrolling

For CSS programming use display:none option to limit social sharing buttons and other extraneous details in the mobile environment.

SOURCE: Campaign Monitor Responsive Email E-Book (2013): http://www.campaignmonitor.com/guides/mobile/


Why focus on Responsive Mobile Emails?

Email Invitation on iPhone

Survey that appears after tapping “Yes”


Mobile Optimized and Non-Optimized Emails

SOURCE: Campaign Monitor Responsive Email E-Book (2013): http://www.campaignmonitor.com/guides/mobile/design/

Mobile Optimized Not Mobile Optimized

http://www.campaignmonitor.com/guides/mobile/

http://www.campaignmonitor.com/guides/mobile/design/


Example of Responsive Email from Amazon.Com

Email sent to Amazon Customers Viewed on

Desktop/Laptop (responsive)

Same email viewed on

Smartphone

Source: http://marketingland.com/four-responsive-email-layouts-15858

Column Dropping


From Responsive Email to App-Like Mobile Survey…

http://marketingland.com/four-responsive-email-layouts-15858


From Email to Mobile Web…


Responsive Email Quick Survey (Square, 2014)


Example of an Email Invitation – Fresh off the Servers!

SCROLL


Internet Use, Time of Day and Device, US

Source: comScore, 2013 http://bit.ly/1c9vneo :

http://www.comscore.com/Insights/Presentations_and_Whitepapers/2013/2013_Mobile_Future_in_Focus


Source: comScore, 2013 http://bit.ly/1QjphfK:

Internet Use, Time of Day and Device, UK

Source: comScore, 2013 http://bit.ly/1QjphfK


Resources for Mobile Optimized Emails

Smith (2013) “Responsive Email: Avoid Defections By Improving Mobile Experiences”

Provides an overview of three common approaches to optimize emails for reading on mobile devices

Mobile Friendly, Mobile Optimized and Responsivehttp://marketingland.com/responsive-email-creating-better-mobile-inbox-experiences-63014

Studabaker (2012) “Four Responsive Email Layouts”Examples of responsive email invitations from major vendors.http://marketingland.com/four-responsive-email-layouts-15858

Studabaker (2012) “Choose The Best Responsive Email Layout For Your Message”

Offers examples of typical layout patterns that are commonly used in responsive email templates

Shrink Wrap, Column Drop and Layout Shiftinghttp://marketingland.com/responsive-email-layout-patterns-29378

http://www.comscore.com/Insights/Presentations_and_Whitepapers/2013/2013_Mobile_Future_in_Focus

http://marketingland.com/responsive-email-creating-better-mobile-inbox-experiences-63014

http://marketingland.com/four-responsive-email-layouts-15858

http://marketingland.com/responsive-email-layout-patterns-29378


Mobile Recruiting Survey Protocols

Once you have optimized the email content and message, when should the email be delivered?

32% of mobile users often read email on their smartphone during the weekend; 31% sometimes reads their mobile email on weekends

54% reads email on their smartphone just before they go to bed sometimes or even more often

49% reads email on their smartphone immediately when they wake up sometimes or even often.

Source: Apsis “The email Barometer, Email on mobile devices” (2013)http://www.emailmonday.com/mobile-email-usage-statistics


Section 6: Smartphone

Survey Data

Collection and

Design Recommendations

16:59


Smarter Smartphone Surveys Development/Design

Recommendations (S3DR)

These recommendations come from our experience in developing surveys for smartphones from the java/html frameworks directly and are based on extensive review of the existing literature and practice.

Some of them are based on the current technical and screen characteristic (i.e. size) of mobile devices –aspects likely to change over time.

The design suggestions presented here are our recommendations meant as guidance not guideline.

107


S3DR #1: Paradata

Successful implementation of multimode surveys requires

that the survey software include comprehensive device detection

functionality. Device detection is needed to automatically identify

the type and brand of each respondent’s device, and then display

the survey according to that device’s specifications.

-Cazes, et al. Kinesis Mobile Landscape Report, 2010http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

Plan to collect paradata from mobile sample including device type, size and browser size. Browsers matter and can be key to an overall survey’s functionality on a respondent’s device.S3DR #1

http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf


User agent strings

“A text field in an HTTP request header that contains the name and version of the Web browser” (PC magazine definition)Web resource for locating your User Agent Strings: http://user-agent-string.info

User agent strings look like this:Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-

US) AppleWebKit/534.7 (KHTML, like Gecko)

Chrome/7.0.517.41 Safari/534.7

Mozilla/5.0 (iPhone; U; CPU iPhone OS 3_1_2 like Mac

OS X; en-us) AppleWebKit/528.18 (KHTML, like Gecko)

Version/4.0 Mobile/7D11 Safari/528.16

BlackBerry9700/5.0.0.351 Profile/MIDP-2.1

Configuration/CLDC-1.1 VendorID/123

109


Device Feature Detection –

Moving Beyond User Agent Strings

Sorting out the contents of User Agent Strings Resource:http://blog.trasatti.it/2010/10/sorting-user-agent-strings-out.html

In addition to phone types and operating systems, features of devices/browsers can be obtained through various device description repositories (DDR) maintained by third parties and could be used in advance to determine which information (i.e. javascript, HTML markup, etc) to send to the phone to render the mobile survey page.

Device Atlashttps://deviceatlas.com/device-data/devices (Browse library of information available on each phone model, for example).www.deviceatlas.com (User Agent String tool)

WURFL (Wireless Universal Resource FiLe)http://wurfl.sourceforge.net/

http://user-agent-string.info

http://blog.trasatti.it/2010/10/sorting-user-agent-strings-out.html

https://deviceatlas.com/device-data/devices

http://www.deviceatlas.com/

http://wurfl.sourceforge.net/


Using RDRs in action – WURFL Example



Beyond User Agent Strings Feature Detection

Feature detection is often deployed using a JavaScript library like modernizr (www.modernizr.com) and is done through a series of queries made to the phone.

If you are interested or concerned about tracking mode effects (especially primacy or recency effects), knowing the type of device/operating system may not be enough.

Keep in mind that while most smartphones (and tablets) use the native browser (safari for iDevices and Android Browser for Android devices) users can and do change these defaults.

Interpretation of the survey scripts can be interpreted differently across browsers and some functionality will gracefully degrade or not be offered as intended

number entry presented as text; photo upload not offered


http://www.modernizr.com/


Browser Types on Desktops (April 2014- April 2015)

Source: StatsCounter: GlobalStats (2015) http://gs.statcounter.com/

A closer look at the Variability in Browsers across device types using desktop internet traffic data collected over the past year (from Stats Counter)


Browser Types on Smartphones

(May, 2014 – April, 2015)

Source:StatCounter GlobalStats (2015) http://gs.statcounter.com/


Here’s a look at why browsers matter

Ultimately, device detection via user agent strings helps to deliver overall content to the correct type of device (phone versus tablet versus computer).

However, the way the device translates the survey programming (HTML markup etc) is determined by:

The browser in useThe features of the browser The features of the device

(e.g. hard keyboard versus virtual, front facing camera or not)


Same view – different view!

When a media query is issued for the size of the viewport for each of these smartphone the result is 320 pixels. However, each of these phones have extremely different browsers with different capabilities.

Source: http://www.slideshare.net/yiibu/pragmatic-responsive-design


Self-Demo 1:

Explore Your Device’s UA String and Size

Using the following web-resources, determine your Smartphone’s user agent string and screen size.

User Agent String:http://user-agent-string.info

Screen Size:http://whatsmy.browsersize.com/


Specific Example: Same Phone…Different Browser

http://user-agent-string.info/

http://user-agent-string.info

http://whatsmy.browsersize.com/


Why browsers matter for Mobile Surveys!

Android Browser (default/native)

Galaxy Note 3 Android (KitKat

4.4.2) SmartphoneChrome Mobile

Browser (activated)


HTML5 and the rise of app-like functionality

The advent of HTML5 has given programmers and researchers a lot of flexibility and versatility in creating mobile versions of websites with many supported input types for mobile forms and capabilities:

Geolocation collectionMedia capture (voice, photo and video)Improved and expanded input typesWeb storage

For survey researchers, this new development means that our active mobile browser surveys can be made more and more app like with standard, predefined enhancements that have been include in HTML5.But this is not yet a silver bullet!! Care must be taken!

http://html5test.com/index.html


HTML 5 functionality across mobile browsers

Mobile HTML 5 Resources are available to determine if certain functionality of interest is available on a particular mobile browser.

http://mobilehtml5.org/http://caniuse.comhttp://detectmobilebrowsers.com/


Alternate HTML 5 Browser Functionality Tool

Another comprehensive tool for tracking availability of HTML5 functionality across browser types is http://caniuse.com

http://mobilehtml5.org/

http://caniuse.com/

http://detectmobilebrowsers.com/

http://caniuse.com/


Case Study Example of HTML 5 in Action

Walmart in partnership with Mattel Hot Wheels launched a loyalty program for shoppers.Shoppers who purchased Hot Wheels products could receive points redeemable for various rewards by scanning their receipts showing the Hot Wheels item purchased. The loyalty program required NO app, yet enabled mobile photo capture!

Scanning/photo functionality and uploads to the Walmart program were managed via an online website optimized for mobile visitors with (modern) browsers


Hot Wheels Redemption Program on iPhone

SOURCE: Hot Wheels (2013) https://hotwheelsrewards.com/

https://hotwheelsrewards.com/desktop/


Practical Example from New Study!

Michaud, Buskirk and Saunders (2014) designed a study in partnership with Decipher and Research Now to compare various touch and data collection possibilities across three modes –computers, tablets and smartphones.A photo capture question was deployed to all devices detected as either “Tablet” or “Smartphone” using user agent string classification.Kindle Fire Tablets Respondents running the native Silk browser had NO photo uploads as expected from browser limitations

Those with devices without this functionality had to press continue twice to continue past this screen.


S3DR #2: Mobile Survey Approach

Select an implementation approach that:(a) optimizes presentation of

survey content (b) is consistent with the type of

survey data desired (c) is appropriate for the context.

S3DR #2


Share of Demographic Audiences by Platform

Source: comScore Media Metrix Multi-Platform, U.S., Age 18+, Dec 2014 / Dec 2013

http://bit.ly/1D2zXhQ


Focusing on Digital Double and Triple Vision

Source: Pew Research, 2012 http://bit.ly/1ttquIX

Source2: comScore (2013) http://bit.ly/1msbiaV

Percentage of Specific Tablet Brand Owners who own Smartphone of a given brand

http://bit.ly/1D2zXhQ

http://bit.ly/1ttquIX


Not all Tablets are Created (and used) Equally

Source: http://www.statista.com/chart/2075/how-different-tablet-users-spend-their-time/


Daily Social Networking Visits and Sharing by Country

41

6153 56

5256 53 56

1814

2326

1612

19 22

0

10

20

30

40

50

60

70

Germany Sweden UK USAVisiting Social Networking Sites (via smartphone)Visiting Social Networking Sites (via Computer)Social Sharing via SmartphoneSocial Sharing via Computer



Tablet versus Smartphone Usage by Task (European Technographics Consumer Technology Online Survey)

Source: Forrester Research, 2012: See: http://tcrn.ch/1SHLGaO


Sweden Internet Users, TV + 1 Device

Source: Inizio “Multiscreen 2013” See: http://bit.ly/1FJyMSW

http://bit.ly/1FJyMSW


While IOS devices update frequently, Android device updates are initiated via Cell Phone Providers and can often lag behind currently available versions.

The functionality of smartphone components such as camera, scanners etc. will in part be governed by both device and operating system.

By using the older version of the OS to develop the app, the functionality will adapt in newer phones and not excluded from older devices.

Moreover, your survey app will be “found” in Market searches across operating system versions.

S3DR#2b

S3DR #2b: Survey Apps Should be Developed using versions

of the Toolkit that are at least one-version back from current


Global Android version distribution since Dec. 2009

Source: http://en.wikipedia.org/wiki/Android_version_history

Dec. 2010Feb. 2011Oct. 2011June 2012


Thank you!

[email protected]

314-695-1378

www.m-s-g.com

@trentbuskirk


Survey Research using Emerging Technologies:

Trent D. Buskirk, Ph.D.JOS Conference June 10 and 11, 2015

AAPOR Short Course, 2014

Considerations for Design, Data Collection, Sampling

and Recruitment for Smartphone and Tablet

Based Surveys

mailto:[email protected]

http://www.m-s-g.com/


Agenda & Goals


Recommendations for Internet/App based survey solutions (Top 14 EBPs)

Computing – just how do you make these surveys? From Websites to Mobile Web Surveys (Web design methods for Mobile)Overview of “Mobile Versions” for Popular Online Survey Packages

Goal: Leave with a solid appreciation of:Potential role of new technologies in data collection;Different ways you can use smartphones to collect survey dataSome practical “best practices” for deploying mobile surveys

Day

2


Smartphone surveys should be designed to be as short as possible, but not necessarily abbreviated versions of longer online surveys.S3DR #4

S3DR #4: Survey Length

Dead spots still exist for SmartphonesIn both voice and data (i.e. edge, roaming)

Internet speeds vary also by area Sprint has both 4G and 3G speeds in St Louis, for example.

Smartphone users multitaskThree good reasons to allow for users to return to the survey site

Start over?Begin where they left off (approximately)


S3DS #1

The overall survey layout should minimize the need for scrolling (either horizontally or vertically) to the extent possible. Survey layout should also minimize the need for pinching/zooming. The number of questions per screen should generally be determined in order to minimize scrolling (implied scrolling)Be mindful for the device and place key actions in “good” zones

S3DR #5

S3DR #5: Survey Layout

Some devices don’t support zoomingAnswer choice layout may create need for scrolling on some devices.


Putting Responses within Reach, for mobile devices!

Source: http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/

http://www.kickerstudio.com/2011/01/activity-zones-for-touchscreen-tablets-and-phones/


Illustrating an ACTIVE Mobile Web Browser Survey:

Peytchev & Hill Study (2010)

Experiment with 92 adults provided with a Samsung Blackjack phone2.2. inches and resolution of 320 by 240 pixels

Several embedded experiments to test question wording effects, image effects, scrolling effects and open ended effects

141


Horizontal Scrolling Experiment (Peytchev & Hill, 2010)

No Significant Differences Noted

More Responses

More

Questions


Example of Survey from Frontier Airlines

(2014) with Horizontal Scrolling Prompts


The Rotate Screen Prompt has Debuted!


Consider reducing the number of response options for grid questions or reorganizing grid questions as single questions per screen.

Consider organizing response options vertically or horizontally with scale labels provided per occurrence (see Peterson, 2013).

If Multiple choice/select all that apply require a long list of alternatives, consider converting question into free response with as many single-text fields as answers expected (i.e. what are three of your current research interests – list up to three).

S3DR #6

S3DR #6: Question Formats


Some Grid Examples on Mobile Devices

Example of Grids – reworked for mobile surveys

Source: Peterson et al. (2013) http://bit.ly/1i5WYo0


Got Health Apps Study Screenshots Illustrating

Question Layout and Response Choice Feedback

Source: Buskirk and Andrus (2014) http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref


Paging Versus Scrolling Question Presentation

Scrolling question presentation in mobile surveys presents a series of survey items on a single page

Requires respondents to scroll (vertically) in order to see (and answer) all survey questions on the page

Paging question presentation in mobile surveys limits the number of questions per page

Generally one question per page and the respondent navigates multiple pages in order to complete the surveyBuskirk and Andrus (2014) present a hybrid with two questions per page to limit (a) the vertical scrolling required on mobile; (b) the overall number of pages needed for survey completion and (c) increase comparability for desktop version

http://fmx.sagepub.com/content/early/2014/04/08/1525822X14526146.full.pdf?ijkey=jZzKaocZyiG6YNn&keytype=ref


S3DR #7: Scrolling Versus Paging

(a) Consider an optimal combination of these methods for presenting questions.

(b) If you have a series of questions with similar response options and type then scrolling may help move a respondent along more quickly and simply.

(c) If you have multiple skip pattern logic in your survey, paging may be preferred (at least for that section).

(d) Be careful that scrolling is suggested or implied so respondents won’t skip (especially if Next/Back are at the top)

S3DR #7


Closer look at Paging versus Scrolling

Survey Outcomes for Mobile Respondents Paging Scrolling

Ease of Completion (Mavletova and Couper, 2014)

Lower Break-off rates (McGeeney and Marlar, 2013; Mavletova and Couper, 2014)

Shorter Completion Times (Mavletova and Couper, 2014)

Lower Overall Item Nonresponse (Mavletova and Couper, 2014)

Fewer Technical Difficulties when Completing Survey

Greater variety of question types/inputs

Maximizes stored data if internet connection is lost and respondent returns to the webpage for completion

Easy Facilitation of skip pattern logic

Source: http://bit.ly/1nqe6Ib

http://conftool.gor.de/conftool14/index.php?page=browseSessions&presentations=show&form_session=9


An example of Scrolling in practice

Requires tapping here; then entering value; then tapping out

then scrolling and repeating.


Example of a Survey w/ a Heavy Scrolling Section


Sliderbars, sum tallies and drag and drop sorters may not work on all devices. These question types should be tested extensively using emulators prior to their adoption in your mobile survey.Text entry fields should be wide enough so that users can see their entire entries without scrolling horizontally. Use this type of entry judiciously.Decisions about types of input/forms or input should include consideration of number of taps or clicks required (NTR or NCR) to enter (and register) an answer.

S3DR #8

S3DR #8: Question Types


A quick note about sliders

Be wary of the default starting positions – could grossly impact final results

Michaud, Buskirk and Saunders, 2014Peterson et al., 2013Can’t distinguish a “3” from missing if default start is midpoint

Source: Peterson et al. (2013) http://bit.ly/1i5WYo0


Using the NTR metrics to help weigh design

choices, in practice.

Michaud, Buskirk and Saunders (2014) included an experiment to compare radio button and list inputs for various substantive questions.

Initially respondents that were randomized to lists received this format for all survey questions including demographic questions about gender, education, race, etc..

Pilot testing and the NTR metrics changed our plans for the demographic questions…


Using the NTR metrics to help weigh design

choices, in practice.

Radio Button Version (all devices)

List/Dropdown iOSand Stock Android

Selector Android (some browsers)


Beware of the Pagination/Scrolling Mashup!

Just because it’s optimized for smaller screens doesn’t mean it’s optimized for respondents!

Presentation of questions only without their response options will save space on a screen, but makes data collection/completion a mine field! Example from Major League Baseball!

https://www.fansatbat.mlb.com/PORTAL/default.aspx

The survey question (depicted on an iPhone 5): “Are any of these networks included in your current cable lineup?”


Taking Advantage of New Input Types from HTML5

With the release of HTML 5 there are several new input types available that might be of interest to survey researchers and questionnaire designers including:

NumberTelEmailRange (a.k.a. sliders)Time

These new input types when used wisely will likely reduce:The overall amount of additional scripting that is required to format the survey questions and response options

Reduce survey page load timesThe NTR metric for a given question

Source: MobiForge, 2013 http://bit.ly/1aH3vnM

https://www.fansatbat.mlb.com/PORTAL/default.aspx

http://bit.ly/1aH3vnM


Numeric, open ended questions using new HTML

input options Number and Tel

How many miles per year do you drive the car that you personally drive most often. (Enter a number between 0 and 50000)

Question PageDefault Input Using HTML5

“number” input type


Making Numeric Input Similar across Devices

By using the HTML5 “tel” input type, a numeric keypad similar to what is displayed when making calls is displayed on all devices.

Formats consistent across devices in

terms of number entry. Note however, that if

decimals are required, then iOS keypad does not offer this capability

directly.


Labels for Open Ended Fields

Labels Top Aligned Labels Within Fields


Labels for Open Ended Questions Top or Within?

Pros for Labels at Top Cons for Labels at top

Pros for Labelswithin

Cons for Labels within

Minimize completion time Requires more space

Saves vertical space Hint text can be misconstrued as acompleted item

Easier coding (no indents/ tables)

Accessibility: label then field in order

Results in longer pages (requiring scrolling)

Reduces page length for pages with multiple open-ended questions

Depending on taplocation the hint text may not disappear from the respondents answer

Supports multiple screen resolutions

More space for the item (horizontally)

Can save on horizontal scrolling required compared to left aligned labels

Once respondent begins typing, the full question or item isno longer fully visible

Reduces horizontal scrolling

Source1: Wroblewski, 2012 http://bit.ly/1eFrXaG Source 2: Penzo, 2006: http://bit.ly/1iuOFC2


Example of Label Alignment in practice

From Wroblewski, 2012 http://bit.ly/1eFrXaG


S3DS #1

Limited screen landscape requires compromise and judgment when using logos, progress bars, disclaimers and help links in mobile surveys.

Consider using abbreviated progress bars

Placement of next and back buttons should be oriented toward the top of the screen or at the bottom of the screen as long as they are persistent and visible without scrolling

Disclaimers should be placed on the welcome page.

S3DR #9

S3DR #9: Navigation/Progress

http://bit.ly/1eFrXaG


Illustrating Back Button and Progress Bar Placement

Next/Back Buttons

Progress Bars


Another Take on Next/Back and Progress Bars

Strategic Reflections Quick Serve Restaurant Survey, 2014


Another Example of Persistent Progress Bars (Expedia Survey, 2015)


S3DS #1

Response selections, action buttons and navigation tabs in surveys should be made as large as possible on mobile surveys.

Touch input differs from Mouse input.

Fattest Fingers – Average width of index finger is 11 mm with a range of under 7 for babies to over 19 for tall athletes.

Make sure to include padding around important navigation tabs (next/continue/enter) to avoid accidental taps .

S3DR #10

S3DR #10: Buttons, Tabs and More Buttons


Importance of Button Sizes

Make butons/tabs large enough to not be missed/avoid mistaps40 pixels (7 mm by 7mm) GOOD50 pixels (9mm by 9 mm ) BETTER30 pixels (5 mm by 5 mm) OK

Give enough buffer around/between tabs/buttonsPadding of at least 2mm (10 pixels) between targets

Source: Microsoft Touch Guidance


Touch Reference: iPhone IOS Requirements

iOS requires app icons to be 57 square pixels for iPhones

iOS requires app icons to be 72 square pixels for iPads

Knowing the resolution of the devices is also important for rendering. Retina display phones and tablets for example usually have double the size allowance for icons (and images).Check out this icon reference chart by graphic illustrator Vicki Wenderlich:

http://www.vickiwenderlich.com/2012/09/app-icon-size-reference-chart/

http://www.vickiwenderlich.com/2012/09/app-icon-size-reference-chart/


One accidental tap could be missing data!

From Michaud, Buskirk and Saunders (2014)We included slider bars as input options for scale questions. The slider bar button was made large enough to drag across phone and tablet devices.On smartphones such as iPhone, the “continue” tab was too close to the upper answer choices, so people who dragged to either 7 or 8 could have tapped the continue button by accident on the way to a rating of 9 or 10. In the radio button version, people attempting to tap an 8 9 or 10 could have tapped continue on the way there and recorded no data.


S3DS #1

Resolution of graphic images should be reduced in proportion to expected screen resolutions of mobile devices prior to transmission to devices (avoid using strictly responsive only web designs for mobile surveys if these surveys will contain images. Prefer RESS approach or adaptive RWD instead).

Flash content should be limited if iPhone Users are part of sampling frame. Alternate video playing capabilities are emerging through HTML5 but not ubiquitous across devices yet. Video files should be embedded via link to YouTube or like “player” Can’t rely on QuickTime, windows media player, etc.Consider the size of the audio, image and video file sizes as they have implications for both bandwidth and data consumption.

S3DR #11: Flash, Video, Audio, Images

S3DR #11


A quick overview of IMAGE Types with focus on

Mobile Device Surveys

Image Type

Common Use Notes

JPEG Typically used for photographic images A small reduction in quality (80% vs 95%) can reduce overall image weight by more than ½. 50% quality images can render quickly and without visible differences on many mobile devices/websites.

PNG PNG is superior to GIF. It produces smaller files and allows more colors. PNG also supports partial transparency. However not all browsers support this function (IE). PNG files are also the only image type with lossless compression supported across the web.

PNG files of photographs can be bloated in size compared to corresponding jpeg images (5 to 7 : 1) PNG files however can be compressed further and still deliver the same image without loss of detail. Try PNGCrush and OptiPNG or Smush.it http://www.smushit.com/ysmush.it/

GIF If the image has fewer than 256 differentcolors or contains areas of similar color then GIF will be optimal.

For images with greater numbers of color or detail, GIF compression may miss some of the colors. But feature phones may not have a palette that goes beyond the 256 standard gif palette anyway.

WebP Newer format developed by Googlehttps://developers.google.com/speed/webp/docs/webp_study

Can provide more compression of images ofcomparable quality saved as jpeg.


Example of PNG Image Compression

BEFORE: 13.66KB

AFTER: 11.01 KB

http://www.smushit.com/ysmush.it/


https://developers.google.com/speed/webp/docs/webp_study



Additional Issues with Collected Images

Images captured by mobile devices contain meta data that includes:

Camera information (aperture, shutter speed, focal length)Date and Time informationGeo LocationOrientation of the image (portrait or landscape)

This information along with the contents of the photo (respondent plus others who may not have consented to be included in the study) have potential privacy concerns.Consider stripping the collected images of this meta data and save it in it’s intended orientation prior to image upload.

Not only reduces privacy concerns about location but also reduces the overall size of the transferred file if done client side.

http://cloudinary.com/blog/top_10_mistakes_in_handling_website_images_and_how_to_solve_them


S3DR #12: Recommendation for Mobile Surveys that have

a Multilingual Audience/Various Language Versions

Remember the higher incidence of mobile users among Hispanics and Asians in the U.S.

When deciding on spacing of answer choices, resulting font size implications and layout for a given set of questions, make choices in light of the languages that will be presented.

For example, if the Spanish version of the answer choices are actually longer, then optimize for these choices first rather than always optimize for English versions.

S3DR #12

http://cloudinary.com/blog/top_10_mistakes_in_handling_website_images_and_how_to_solve_them


Optimizing for English Versions may not be sufficient!

Questions or their answer choices (in English) that appear to be well optimized and presented using mobile browsers may not be optimally presented in another language

ENGLISH VERSION SPANISH VERSION


S3DS #1

Apps, unlike mobile web browsers, can enable GPS location services via the Smartphone’s GPS Antenna directly.

Geolocation data can also be collected via browsers that support the HTML5 functionality (accuracy varies by cellular and wi-fi connectivity.In IOS GPS permissions are granted/controlled on a “per app” basis; Android devices generally use global controls;Should your survey require GPS data collection (passive), the respondent should be given the option to opt out/in via a push notification at the time of the app launch. Don’t assume that once an opt in is given once that it persists across survey data collection points.

S3DR #13: If GPS location data are required, user

opt-in/out notifications should be pushed

S3DR #13


Geolocation Data via Mobile Devices

Respondents – where art thou?GEOLOCATION is the capability to detect and record where a mobile phone is located using various methods:

IP address, Mac address, RFID, Wi-Fi connection location or GPS coordinates.

GPS or Global Positioning System is a satellite-based navigation system. GPS uses triangulation to calculate a user’s exact location.

Smartphones that are GPS-enabled allow an app to determine a phone’s exact location. Using HTML5, geolocation can be estimated form mobile deices via the mobile web browser (wi-fi versus cellular networks)

GEOFENCING Technology that provides the ability to create a virtual fence around a geographic location in the real world.

Smartphones that are location-enabled detect when someone enters or exits these fences, (Big: city block or Small: retail store).


Geolocation detection through BROWSERS!

A number of different sources are used to attempt to obtain the user’s location, and each has their own varying degree of accuracy.

A desktop browser is likely to use WiFi (accurate to 20m) or IP Geolocation which is only accurate to the city level Mobile devices tend to use triangulation techniques such as GPS(accurate to 10m and only works outside), WiFi and GSM/CDMA cell IDs (accurate to 1000m).Accuracy on mobile phones depends on several factors including:

Whether wifi is activated or not (for android phones this has a huge impact)Is phone in an urban or rural area?Is phone moving or stationary?Is a VPN turned on?http://html5doctor.com/finding-your-position-with-geolocation/http://www.andygup.net/html5-geolocation-api-%e2%80%93-how-accurate-is-it-really/

http://html5doctor.com/finding-your-position-with-geolocation/

http://www.andygup.net/html5-geolocation-api-%e2%80%93-how-accurate-is-it-really/


Geolocation through Browsers

There are some pros and cons for geolocation through browsers that we need to be aware of in order to better use these types of data

Not all browsers support the Geolocation functionality of HTML5 (IE 8 does not, for example).Different browsers use different Geolocation services (mapping)

Chrome browsers=Google Location ServicesSafari browsers on iOS devices=Apple Location Services

Requires and internet connection and explicit user permission (as per HTML 5 Geolocation Guidelines for use). May not be useful for persistent Geo-location tracking or for geo-aware notifications such as used in Geo-fencingDoes not generally have the same drain on batteries as the actual GPS hardware.Does not require an app to be installed in order to identify location of respondent.


Accuracy of Geolocation via Browsers

Accuracy can be impacted by whether or not WiFi is turned on, for example. Here is a test link and the results I obtained with and without WiFi using an iphone 5 (ios6)

http://andygup.net/samples/html5geo/

http://andygup.net/samples/html5geo/


Locately: (http://www.locately.com/)Point of interest matchingTrip AnalyticsData processing (No integrated API)Panel built from anonymous user data

Loc-Aid (http://www.loc-aid.com/)Integrated APIAllows for custom geo-fencing

Smartphone GPS Data Vendors


7-Eleven and am/pm stores that one respondent visits

HomeGym

School

3 Visits:on way to

Gym 1 Visit

4 Visits:on way to

School


There is a great deal of functionality that is consistent across mobile smartphones- but there is enough inconsistency to require extensive pre-testing of your mobile survey across multiple platforms.

Consider using smartphone emulators to pretest the mobile version of your survey or alternatively a virtual device laboratory to test mobile versions of your survey.

Test availability of flash content using a resource such as: http://www.adobe.com/flashplatform/supported_devices/smartphones.html

(or on your mobile phone/emulator) http://www.adobe.com/software/flash/about/

S3DR#14: Pre-Testing Mobile Surveys

S3DR #14


Smartphone Emulators

Suggested Emulators for Popular Smartphones

iPhone: http://iphonetester.com/Android Devices: http://www.addictivetips.com/windows-tips/download-google-android-emulator/Blackberry Devices: http://www.blackberry.com/developers/downloads/simulators/index.shtmlMicrosoft Devices: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=25191

Source: http://sixrevisions.com/web-development/mobile-web-design-best-practices/

http://iphonetester.com/

http://www.addictivetips.com/windows-tips/download-google-android-emulator/

http://www.blackberry.com/developers/downloads/simulators/index.shtml

http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=25191


Mobile Emulator Software Example:Keynote’s DeviceAnywhere

http://www.keynotedeviceanywhere.com/da-free-product-overview.php

Many types of devices centrally

located in one source (host of

tablets and smartphones)


Section 7: Web Design

Approaches

to Mobile

Surveys (DIY)

16:59

http://www.keynotedeviceanywhere.com/da-free-product-overview.php


Active Mobile Browser Survey Development

Three main approaches to developing an active mobile browser survey:

Responsive Web Design (RWD) for Survey PagesMobile-First RWD

Adaptive Designs Responsive Design with Server Side Components (RESS)

Separate Mobile Web site developed for Surveys

A multi-screen strategy is a must to succeed with today’s constantly-connected consumers. But one size doesn’t fit all when it comes to creating the perfect configuration. Many factors come into play, and you know your own business needs best. – Google 2013 http://bit.ly/1i9e3P1 and SURVEY


Responsive and Mobile Responsive Web Design

Responsive web design (RWD) is a method for web design that uses fluid grids, images and media queries to change the displayed layout based on the size of the device (view port size).

Media queries govern when and how the fluid layouts appear on given devices and is managed by the respondent’s deviceSingle HTML source code delivered to all devices (one size fits all)

Mobile-first Responsive Web design is a method that begins with the mobile version first and pushes the web page to respond to larger devices via progressive enhancement.

Additional layout attributes and survey functionality can be added progressively, in real time for respondents accessing website using larger devices.

ReferencesFrost (2011) http://bit.ly/1gOcOPK Passani (2013) http://bit.ly/1i9G8WgWroblewski (2011) http://bit.ly/1gOcS2d

http://bit.ly/1i9e3P1

http://bit.ly/1gOcOPK

http://bit.ly/1i9G8Wg

http://bit.ly/1gOcS2d


RWD and Mobile First RWDs for Surveys

Source: http://web3canvas.com/what-is-mobile-first-responsive-design/

RWD for Surveys would be a strategy that takes an online (computer survey) and thinks about how to render it down to tablet and then down to smartphones.

Mobile First RWD would take a mobile survey and progressively push it to larger devices with potentially different functionality.


Responsive Web Design Example: www.SweetHatClub.org

http://www.sweethatclub.org/


Responsive Web Design Example: www.SweetHatClub.org


Adaptive Designs

There are various versions of Adaptive web designs but the approach that has a lot of traction recently is Responsive Web Design with Server Side Components (RESS)

RESS is sometimes referred to as a hybrid approach between Server-side implementation (rendering unto Caesar what is Caesar’s) and RWD. RESS uses device detection on the server side to send device specific survey web page content/scriptingThe device specific code can include components that allow it to progressively enhance the survey page depending on browser features it detects on the respondent’s device.

References:S. Rieger (2011) http://slidesha.re/1k1HLUH Pratap (2013) http://bit.ly/IK6Q8A

Ronan (2011) http://bit.ly/1tuHNcJ B. Rieger (2011) http://slidesha.re/1pnzBeA

http://www.sweethatclub.org/

http://slidesha.re/1k1HLUH

http://bit.ly/IK6Q8A

http://bit.ly/1tuHNcJ

http://slidesha.re/1pnzBeA


Survey Questions

Images used in Survey

Computer Scripting and

HTML Markup (all devices)

www.takesurvey.com

RESPONSIVE WEB

DEISGN – 10000

Feet Overview


Survey Questions

Images for

PHONESComputer

Scripting and HTML Markup

(PHONES)

www.takesurvey.com

RESPONSIVE WEB DEISGN

with Server Side Components

10000 Feet Overview

What Kind of Device is it? User

Agent String Says:

iPhone

http://www.takesurvey.com/

http://www.takesurvey.com/


Giving your Online Survey Site a Mobile Sibling

Separate mobile version of the site is the ultimate adaptive strategy

Special scripts are created to program survey functionality specifically for the mobile devices

The survey website will generally be a stand-alone website and could potentially require a different web address.

If you know a priori that you are recruiting mobile respondents, then a unique mobile link and QR code could be provided.

If you are interested in field work or in-person data collection using mobile devices and you don’t want to use an app, then a separate mobile web survey optimized for mobile devices could be a cost effective and viable solution.Frost (2012) offers a great comparison or responsive and separate versions of presidential websites from the 2012 election.



Data COLLECTION!


How to choose a survey design approach?

Responsive Web Design ProsAll HTML scripting required for survey deployment across any device is contained in a single packet and is delivered to every device (OSFA)Survey content will be rendered seamlessly across a wide array of devices (even ones that aren’t invented yet or that you didn’t plan for)Single Survey URL serves all devices/sizes/typesSurvey content will automatically adjust to Horizontal or Vertical orientations

Responsive Web Design ConsAll programming code required to render surveys on any device is downloaded to the respondent’s device for processing (unnecessarily large data transmission size and potentially longer rendering times)Images are generally much larger in size than they need to be for mobile devices (again, increased data transferred to phone/tablet than required)Typically does not take full advantage of mobile device functionalitySurvey layout cannot be controlled as easily and in many cases pages will be longer and require more scrolling.

Source: http://bit.ly/QBI4Lv



Mobile First Responsive Web Design ProsBecause programming and layout focus on mobile, images and therefore entire HTML mark-up will be smaller compared to RWD designsMobile functionality can be built in to the survey coding from the ground upTotal number of HTTP requests between the mobile devices and the survey host will be minimized in order to render the same content – thus reducing latency and loading times.Has been used successfully by Survey Researchers to collect data from both smartphone and computer respondents

Bruijne and Wijnant (2013) http://bit.ly/1lxs0UX

Mobile First Responsive Web Design ConsRequires re-engineering online survey website (computer version) from the beginning with mobile surveys as the basis **Survey layout cannot be controlled as easily and in many cases pages will be longer and require more scrolling.

Source: http://bit.ly/1nl50MT



RESS Approach ProsBecause content can be packaged for devices, http requests and overall sizes of markup tends to be smaller (decrease load time and data transferred)Can include components of progressive enhancement to build out features that are detected on respondents device Single Survey URL serves all devices/sizes/typesSurvey content will automatically adjust to Horizontal or Vertical orientations

RESS Approach ConsIn order to send appropriate versions of survey scripts, a device detection data base/repository is required and would require updatingTo maximize information sent from the server and to minimize subsequent http requests for rendering survey cookies can be used

May require informed consent to place cookies (EU Cookie Directive)Server side computing resources and specific programming is required

http://bit.ly/1lxs0UX


How to decide the approach used here?


Resources for Assessing Size and Programming

Features of your Survey Webpages

YSLOW – a downloadable/executable program from Yahoo to provide a grade for your website in terms of Web Performance Best practices

Includes Image Assessment, Overall size assessment and morehttps://developer.yahoo.com/yslow/

PageSpeed – executable online program from Google to assess page performance and user experience criteria based on Web performance best practices

Online version includes Image Assessment, overall size, speed and user experience criteria for both MOBILE and ONLINE with suggestions for improvementshttp://developers.google.com/speed/pagespeed/insights/

https://developer.yahoo.com/yslow/

http://developers.google.com/speed/pagespeed/insights/


A Worldwide Wonderland for the World Wide Web

(Mobile included!)

Responsive Web Design Example: www.disney.com

mini


Let’s Look Under the “Hood” of the Disney Site.

An example of a bloated design includes background images (skin) along with many foreground images. www.disney.comThis took about 10 seconds to load using my iphone 5 over the LTE network of Verizon

http://developer.yahoo.com/yslow/

http://www.disney.com/

http://www.joyintermedia.com/


From Google PageSpeed: Disney.com


The other side of RWD…LAPTOP Pinch and Zoom!

Responsive web design can be great at adapting content to smaller viewports (or devices) but one of the drawbacks for surveys is the bringing back the pinch and zoom issue to lap/desktops as illustrated here – a laptop and iphone view of a question from a sample survey from CheckMarket. https://www.checkmarket.com/mobile-surveys/

Laptop View

iPhoneView

https://www.checkmarket.com/mobile-surveys/


Focus on a survey design that is technically sound

On mobile websites there is a smaller margin of error for:Image rendering, browser requests, survey page size, waiting times.

77% of cell internet users say they experience slow download speeds that prevent things from loading as quickly as they would like. Of those cell internet users, 46% face slow download speeds weekly or more frequently. (Pew, 2012)97% of time on a single web page is spent on the front end – meaning the loading time of the page (Everts, 2011)

Why do we need to focus on techniques that can optimize survey websites?

Overall Respondent burdenLag/Latency Waiting time, Time on Tasks, data consumption/exchange, effort

Break-off, partial completion, full completionKeep in mind that a web survey is a series of consecutive and linked web pages (each serving up survey tasks to complete).


Practical Tips for Reducing Respondent Device Burden

There are several practical guidelines to think about for reducing overall respondent burden Image Resizing

Deliver smaller images to devices with smaller resolutionsAvoids having to transmit entire image to device and then resize at the device level

Device Specific Scripting OnlyIf displaying survey contents to a smartphone, don’t send deskop/tablet specifications, additions or modifications, for example

Connectivity DetectionNot only for cellular networks, but also for weak wi-fi networks (i.e. laptops accessing weak signal @ airports or conference hotels)


Section 8: Making it

Work! Software

Considerations

16:59


Development Frameworks and Resources

JQuery Mobile: (http://jquerymobile.com/) mobile framework that allows you to design a single web site or app that will work on all popular smartphone, tablet, and desktop platforms.

Pelletier’s E-book: Mobile App Manual: The Blueprint? Provides an introduction to developing apps using J-Query Mobile and the PhoneGapbuild environment. User friendly overview of the approach rather than emphasis on the computer code required. http://mobileappmanual.com/

Sencha Touch: (www.sencha.com) A comprehensive development environment for mobile apps that uses the HTML5 platform

Pure from Yahoo: http://purecss.io/This collection of CSS modules enables the users to create responsively designed websites (for a myriad of devices) using an entire collection of CSS pages that have been designed to be as small as possible to avoid bloating and increased loading/unpacking times on mobile devices.

http://jquerymobile.com/

http://mobileappmanual.com/

http://www.sencha.com/

http://purecss.io/


Software Services Solutions for Mobile Survey

Design/Development/Deployment

WebFirst’s PhiCollect (Paid) www.webfirst.com/phicolect

Open source data collection platform – flexible, scalable, cloud-basedWeb, Android, iOS and SMS options (Tablet and Phone)

Location, multimedia, sms data collectionHas been deployed for mHealth and education applications worldwide.

Qualtrics (Paid)http://www.qualtrics.com/research-suite/mobile-surveys/

Provides feedback on survey questions and answers and formats with recommendations for better mobile optimization. Uses responsive web design with server side components

allows customization of content delivery based on device type detection: ipad, android phone, palm, windows mobile, etc.


Example from a Basic Qualtrics Survey

http://www.webfirst.com/phicolect

http://www.qualtrics.com/research-suite/mobile-surveys/


Software Services for Mobile Surveys, Cont.

Columbia University Modi Research Group’s formhub (Free) http://www.formhub.org/

Python/Django Open source, free development platform for mobile data collectionRelies on ODK Collect app or mobile browser (modern browser that supports HTML caching is required)Surveys are designed and managed using Microsoft Excel XLSForms spreadsheets

One tab is questions; the other is answer optionsSurveys are presented more like data completion forms (scrolling) and multiple completes can be managed from a single device (Tablet and Mobile Phone Assisted Interviewing)

Formhub Demo Survey Available athttp://bit.ly/1nvbXbj


The Actual Survey in FormXLS

http://www.formhub.org/

http://bit.ly/1nvbXbj


Software Services for Mobile Surveys, Cont.

CheckMarket Surveys (Paid)https://www.checkmarket.com/Company headquartered in Belgium and offers responsive-like design for mobile surveys that has some app-like functionality.

Their mobile solution has implemented several of the emerging best practices and provides some flexibility to the survey designer (controlling questions per page, size of windows for open ended questions)New data collection techniques for media are available as well as a ranking question type that is implemented smoothly in the mobile environment

Example Demo Survey is available here:http://bit.ly/1K3hzqu


App-Based Survey Solutions

Confirmit’s SODA (Survey on Demand Application) App. (Paid)Probably the most widely known and used App-based survey solution among the survey research community.Version 3.0 was released in January, 2014

Continuous GPS tracking, Push Notifications, Multiple field editing enhancementshttp://bit.ly/1lllwfQ

Conduit Mobile (Paid)Offers app development in a hosted environment.Can build your own app and can include data collection forms and formats using their web-based app-building platformhttp://mobile.conduit.com/

https://www.checkmarket.com/

http://bit.ly/1K3hzqu

http://bit.ly/1lllwfQ

http://mobile.conduit.com/


The Future of Survey Research is Coming: Fast!

The percentage of unintentional mobile respondents has reached arguably non-negligible levels and will likely increase as email and internet activity on mobile devices increases.

There is still a lot to learn about usability and design considerations for surveys optimized for Smartphones.

Even if you are using a survey platform that can handle mobile devices you will still have to make many decisions in terms of questionnaire design.

Be mindful of variability across platforms- iPhone release may not be sufficient for all types of Smartphones associated members of your sampling frame.

Here’s Hoping that All our Surveys Continue to Get Smarter…

219


Final Comment!

I leave you with this final word from Sir Richard Branson to encourage all of you to continue to:

invest in experimentation in the mobile survey arena and share your results

so that we can continue to develop and use best practices for these very cool (and smart) devices!


Thank you!

[email protected]

314-695-1378

www.m-s-g.com

@trentbuskirk


References

AAPOR Cell Phone Task Force Report, 2010 available at: http://www.aapor.org/Cell_Phone_Task_Force_Report.htm , accessed on October 28, 2011Attfield, Simon, Gabriella Kazai, Mounia Lalmas, and Benjamin Piwowarski. 2011. “Towards a Science of User Engagement (Position Paper).” Presented at the WSDM Workshop on User Modelling for Web Applications, Hong Kong, China.Bailey, Justin and Lorelle Vanno. 2011. “Market Research Association Current Mobile Research Trends.” Webinar for the Florida Chapter of Marketing Research Association.Bosnjak, Michael, Gottfried Metzger, and Lorenz Gräf. 2010. “Understanding the Willingness to Participate in Mobile Surveys: Exploring the Role of Utilitarian, Affective, Hedonic, Social, Self-Expressive, and Trust-Related Factors,” Social Science Computer Review 28: 350-370.Brick, J. M., P. D. Brick, S. Dipko, S. Presser, C. Tucker, and Y. Yuan. 2007. Cell phone survey feasibility in the U.S.: Sampling and calling cell numbers versus landline numbers. Public Opinion Quarterly 71 (1): 23–39. Buskirk, T.D., Callegaro, M. and Rao, K. (2010) ‘N the Network'? Using Internet Resources for Predicting Cell Phone Number Status. Social Science Computer Review: Special Issue on Mobile Surveys, Vol. 28, No. 3, 271-286.Buskirk, T.D. and C. Andrus. (2012a). Smart surveys for smartphone: exploring various approaches for conducting online mobile surveys via smartphones. Survey Practice. Available at: http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/ .Buskirk, T. D. and Callegaro, M. “Cell Phone Sampling,” in Encyclopedia of Survey Research Methods, Lavrakas, P.J. (editor), Sage, Los Angeles, © 2008, pp. 84-89.Buskirk, T.D., Gaynor, M., Andrus, C. and Gorrell, C. (2011) “An App a Day Could Keep The Doctor Away: Comparing Mode Effects for a iPhone Survey related to Health App Use.” American Association of Public Opinion Research, Phoenix, AZ.

mailto:[email protected]

http://www.m-s-g.com/

http://www.aapor.org/Cell_Phone_Task_Force_Report.htm

http://surveypractice.wordpress.com/2012/02/21/smart-surveys-for-smart-phones/


References (con’t)

Buskirk, T.D. and Andrus, C. (2014) “Making Mobile Surveys Smarter: Results from a randomized experiment comparing online smartphone and desktop/laptop computers,” Field Methods, forthcoming. See http://fmx.sagepub.com/content/early/recentCallegaro, Mario. 2010. “Do You Know Which Device Your Respondent Has Used to Take Your Online Survey?” Survey Practice, December: www.surveypractice.org.Callegaro, M. and Macer, T. (2011) “Designing Surveys for Mobile Devices: Pocket-sized Surveys that Yield Powerful Results,“ Shortcourse Presented at the 66th Annual Meeting of the AmericaonAssociation of Public Opinion Research, Phoenix, AZ.Cazes, J., Townsend, L., Rios, H., & Ehler-James, J. (2010). The mobile survey landscape - Today and Tomorrow. Impacts of mobile devices usage on current and future market research practices. Retrieved from http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdfcomScore (2012) 2012 Mobile Future in Focus. Retrieved from http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_FocusCouper, M. P. (2010). Visual design in online surveys: Learnings for the mobile world. Presented at the Mobile Research Conference 2010, London. Retrieved from http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdfCunningham, JA et al. (2013) “Use of mobile devices to answer online surveys: implications for research,” BMC Research Notes. Vol 6:258 available online: http://www.biomedcentral.com/1756-0500/6/258Duan, Shu, Justin Bailey, and Michael Link. 2011. “Data Collection Method Innovation: A Crowdsourcing Application to Collect Factual Information in Africa”. Paper presented at the Midwest Association for Public Opinion Research Conference, Chicago, IL.


Fuchs, Mark, 2008. “Mobile Web Surveys: A Preliminary Discussion of Methodological Implications”. In F. Conrad & M. Schober (Eds.), Envisioning the Survey Interview of the Future (pp. 77-94). Hoboken, NJ: Wiley.Graham, P, and C. Cobb (2013). “Comparison of Instantaneous Mobile Time Use Data Collection Methods to Traditional Time Diary Methods.” Paper presented at the Annual Conference of the American Association for Public Opinion Research, Boston, MA.Gregoski, Mathew, Martina Mueller, Alexey Vertegel, et al. (2012). “Development and Validation of a Smartphone Heart Rate Acquisition Application for Health Promotion and Wellness Telehealth Applications.” International Journal of Telemedicine and Applications, doi:10.1155/2012/696324. http://www.hindawi.com/journals/ijta/2012/696324/Kennedy, C. (2007), Evaluating the Effects of Screening for Telephone Service in Dual Frame RDD Surveys, Public Opinion Quarterly, Vol 71 / Num 5: 750-771.Koch, P-P. (2010) Smartphone Browser Landscape, A List Apart Article No. 320, available at http://www.alistapart.com/articles/smartphone-browser-landscape/ (accessed on 10/30/2011)Kumar, A. (2012) “Pilot study tests wearable sensors and smartphone-based data collection for diabetes,” iMedicalApps, accessed at http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone-diabetes/Kuntsche, Emmanuel and Benjamin Robert. 2009. “Short Message Service (SMS) Technology in Alcohol Research – A Feasability Study.” Alcohol &Alcoholism 44: 423-428.Lai, Jennie W., Lorelle Vanno, Michael Link, Jennie Pearson, Hala Makowska, Karen Benezra, Mark Green. 2010. “Life360: Usability of Mobile Devices for Time Use Surveys” Survey Practice, February: www.surveypractice.org

224


http://fmx.sagepub.com/content/early/recent


http://www.kinesissurvey.com/files/MobileSurveyLandscape_KinesisWhitepaper.pdf

http://www.comscore.com/Press_Events/Presentations_Whitepapers/2012/2012_Mobile_Future_in_Focus

http://www.mobileresearchconference.com/uploads/files/MRC2010_Couper_Keynote.pdf

http://www.biomedcentral.com/1756-0500/6/258

http://www.hindawi.com/journals/ijta/2012/696324/

http://www.alistapart.com/articles/smartphone-browser-landscape/

http://www.imedicalapps.com/2012/11/pilot-study-wearable-sensors-smartphone-diabetes/




Lai, Jennie, Michael Link, and Lorelle Vanno. 2011. “Like? The Effect of Social Networking in the Digital Age of Mobile Application Research.” Paper presented at the Midwest Association for Public Opinion Research Conference, Chicago, IL.Maxl, E., Döring, N., & Wallisch, A. (Eds.). (2009). Mobile market research. Köln: Herbert Von Halem Verlag. Nitsche, P, et al. (2012) “A strategy on how to utilize smartphones for automatically reconstructing trips in travel surveys,” Procedia-Social and Behavioral Sciences, Vol. 48, pp. 1033-1046.Olson, K, and J. Wagner (2013). “A Field Experiment Using GPS devices to measure Interviewer Travel Behavior.” Paper presented at the annual conference of the American Association for Public Opinion Research, Boston (May).Paudela, D., M. Ahmedb, A. Pradhanc, R. Lal Dangold (2013) “Successful use of tablet personal computers and wireless technologies for the 2011 Nepal Demographic and Health Survey,” Glob Health Sci Pract August 12, 2013 vol. 1 no. 2 p. 277-284; http://www.ghspjournal.org/content/1/2/277.full.pdf+htmlPerkins,J, R. Granger, and E. Saleska. 2009. “Data Security Considerations When Using Social Networking Wesites for Locating and Contacting Sample Members.” Presented at International Field Directors & Technology Conference.Peytchev, Andy and Craig A. Hill. 2010. “Experiments in Mobile Web Survey Design: Similarities to Other Modes and Unique Considerations,” Social Science Computer Review 28: 319-335.Pew Research Center. 2011. “How People Use Tablets and What It Means for the Future of News.” Report. Available at: www.pewresearch.org/pubs/2119/tablet-newsScagnelli, Jeff, Justin Bailey, Michael Link, Hala Moakowska, and Karen Benezra (2012). “On the Run: In the Moment Smartphone Data Collection.” Paper presented at the annual conference of the American Association for Public Opinion Research (Orlando, FL).



Schober, Michael, Frederick Conrad, Christopher Antoun, Alison Bowers, Andrew Hupp, Huiying Yan (2013). “Conversational Interaction and Survey Data Quality in SMS Text Interviews.” Paper presented at the Annual Conference of the American Association for Public Opinion Research, Boston, MA.Smith, Aaron. 2011. “Smartphone Adoption and Usage.” Report available at: www.pewinternet.org/reports/2011/Samartphones.aspxSteeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field Methods, 19, 59–75.van Heerden, A. Shane A. Norris, Stephen M. Tollman, and Linda M. Richter, “Collecting Health Research Data: Comparing Mobile Phone-Assisted Personal Interviewing to Paper-and-pen Data Collection,” in press, Field Methods, http://fmx.sagepub.com/content/early/recentVanno, Lorelle, Jennie Lai, and Michael Link. 2011. “Want to Collect Behavioral Data in an Engaging Way? There’s an App for That!” Paper presented at the Southern Association for Public Opinion Research Conference, Raleigh, NC.Zahariev, M., Ferneyhough, C., Ryan, C., & Bishop, S. (2009). Best practices in mobile research. Presented at the ESOMAR Online Research, Chicago, IL: ESOMAR.Zickuhr, K. (2011). Generations and their gadgets. Pew Internet & the American Life Project. Retrieved from http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspxZuwallack, R. (2009) Piloting Data Collection via Cell Phones: Results, Experiences, and Lessons Learned, Field Methods, Vol. 21, No. 4, pp. 388–406Zichermann and Cunningham (2011) Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps available at: http://www.amazon.com/Gamification-Design-Implementing-Mechanics-Mobile/dp/1449397670Andbergen, P. A. (2012). Comparison of WiFi positioning on two mobile devices. Journal Of Location Based Services, 6(1), 35-50. doi:10.1080/17489725.2011.630038

http://www.ghspjournal.org/content/1/2/277.full.pdf+html

http://www.pewresearch.org/pubs/2119/tablet-news

http://www.pewinternet.org/reports/2011/Samartphones.aspx

http://fmx.sagepub.com/content/early/recent

http://pewinternet.org/Reports/2011/Generations-and-gadgets.aspx

http://www.amazon.com/Gamification-Design-Implementing-Mechanics-Mobile/dp/1449397670

Survey Research using Emerging Technologies - m-s-g.com · 2015-08-06 ·...

Documents

Transcript of Survey Research using Emerging Technologies - m-s-g.com · 2015-08-06 ·...