Making it fit: How survey technology proviers are responding to the challenges of handling web...

Making it fit

How survey technology providers are responding to the challenges of handling web surveys on mobile devices

Tim Macer, meaning ltd, London, UKASC Sixth International Conference, Bristol, 22-23 September 2011

2

Agenda

1. Emerging best practices

2. Requirements identified

3. How are the vendors responding

4. Conclusions and recommendations

Making it fit: web surveys on mobile devices. ASC 2011

1. Emerging best practicesSpecific considerations

Operating systems

App or browser?

3


Multi-mode parallel surveys on web/mobile

Design the survey around single-choice, multiple-choice and open-ended questions alone.

Adopt more concise wording

Limit the survey length to fifteen questions.

Zahariev et al (2010)

4


Specific technical requirements identified…

Identifying which specific smartphones are supported, as only specific makes or models may be supported in some packages.

Verifying that error messages and other components are not dependent on the use of JavaScript.

Having a specific ‘skin’ or page template to format the survey for delivery on a mobile device, which maximised the limited space available for questions.

Zahariev et al (2010)

5


Recommended optimiziations for mobile surveys

Use a concise, well-worded SMS invitation with an integrated link to survey.

To ensure any costs a participant may incur are reimbursed

Avoid complex matrix questions: replace with questions presented one by one.

Encourage active engagement, e.g. taking photographs.

Pick time to send invites in order to optimize response (e.g. between 17.00 and 18.00).

Pferdekaemper (2010)

6


Further recommendations for mobile

Reduce and simplify – avoid horizontal scrolling,

reduce clutter from logos, branding,

consider using drop-down lists rather than radio buttons

Luck (2011) “How to make mobile surveys work for everyone”

Adjust the survey to fit limited the screen of each device Understand the dimension of the device

Capture the browser agent string

Callegaro (2010) “Do You Know Which Device Your Respondent Has Used to Take Your Online Survey?”

7


Current trends in Mobile operating systems in use

OS 2010 2014 trend

Android (Google) 17.7 29.6

iOS (Apple) 15.4 14.9

Research In Motion (BlackBerry) 17.5 11.7

Symbian 40.1 30.2

Windows Phone 4.7 3.9

Other Operating Systems 4.7 9.6

8

Mobile Communications devices: sales to end users by OS Gartner (2010)


What defines an app or a browser survey?

browser

A web survey which has been

formatted so that it will work

effectively in the browser of a

range of mobile devices within

the constraints of these

devices.

app

A survey delivered through a

dedicated application or app

which the participant must first

download onto his/her device

and then accesses via an icon

on the device’s main menu.

9


Browser pros and cons

Uses HTML: achieve high coverage; not restricted to a subset of devices

Participant can engage immediately without having to download anything

Mix formats within one survey instrument: laptop/desktop, tablets, handhelds

✘ Lowest common denominator approach to question types

✘ Not all devices support flash

✘ Integrating media, e.g. uploading photos, can be too difficult for participants to do

✘ Needs a stable internet connection each time “next” is pressed

10


App pros and cons

Does not need a permanent stable internet connection at all times

Can exploit all the programmable capabilities of the device, e.g. diaries with reminders, integrated capture and upload of pictures and video

✘ Download requires high level of trust and engagement from participant

✘ Apps rarely support the entire range of devices out there – reach less of the population

11


2. Identifying the key requirementsEssential needs for browser tools

Essential needs for apps

The survey

12


Essential requirements examined: browser-based tools

1. Automatically detect the device resolution and reformat survey to fit

2. Optimize the presentation of the survey for small-format devices

3. Gracefully unwind grids into a series of individual questions

4. Record device type used in the data

5. Integrated survey invitations and reminders by email and SMS

6. Allow respondent to save and resume later in ‘desktop’ or mobile mode

7. No dependency on mouse controls (shift+click etc)

8. No dependency on using Flash controls

9. Can create specific routing and validation dependent on device type

13


Essential requirements: app-based tools

1. Simple, easy download for respondent

2. Web server-based authoring and deployment environment

3. Capture location data, with respondent permission

4. Capture and upload photos and video

5. Diary surveys: repeated capture of same questions

6. Diary reminder alerts with do not disturb option

7. Online/offline capability

8. Bar code scanning with respondent confirmation

14


…plus, requirements common to both modes

10. Wide range of devices supported

11. Device-specific preview in authoring tool or via simulators

15


Contacted 10 different

technology producers

active in the field

Contacts made in

April 2011 with a simple

electronic questionnaire

7 firms with browser-based solutions

6 firms with apps

5 with both

Eight companies

participated

One refused

One failed to respond

The survey


3. How are the vendors respondingPlatforms supported

Features provided

17


Who is providing what?

Supplier Browser App

Askia ✓ ✓CfMC WebSurvent ✓Confirmit ✓Globalpark EFS ✓ ✓iPinion Surveys ✓ ✓Kinesis ✓ ✓Surveyswipe & Surveypocket ✓ ✓Techneos ✓

18


Browser tools – devices supported

Device iPinion KinesisSurvey- swipe

CfMC AskiaCon-firmit

Global-park

Android ✓ ✓ ✓ ✓ ✓ ✓ ✓

BlackBerry ✓ ✓ ✓ ✓ ✓ ✓ ✓

Symbian ✓ ✓ ✓ ✓ ✓

iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓ ✓

Windows mobile ✓ ✓ ✓ ✓ ✓ ✓ ✓

WML ✓ ✓

XHTML ✓ ✓ ✓ ✓ ✓

19

✓ Capability currently offered Planned for a Future release


Mobile Apps – devices supported

iPinion KinesisSurvey-pocket

TechneosSODA

Askia mobile

Global-park

Android ✓ ✓ ✓ ✓

BlackBerry ✓ ✓ ✓

Symbian ✓

iOS (Apple) ✓ ✓ ✓ ✓ ✓ ✓Windows mobile

✓ ✓

20

✓ Capability currently offered Planned for a Future release


Web browser solutions

Analyzing the capabilities offered in the market

21


Browser tools – capabilities offered by all vendors

Capability

Mixed mode – full screen and mobile ✓

Detect and reformat to fit device ✓

Optimize for mobile ✓

No dependency on mouse ✓

No dependency on Flash ✓

22


Browser tools – capabilities where support varies

Capability iPinion KinesisSurvey- swipe

CfMC AskiaCon-firmit

Global-park

Unwind grids ✓ ✓ ✓ ✓ ✓ ✓Store device data

• At survey level ✓ ✓ ✓ ✓ ✓ ✓• At question level ✓ ✓ ✓ ✓Invitations, reminders by Email and SMS ✓ ✓ ✓ ✓ ✓ ✓

Preview via simulator ✓ ✓Save and resume ✓ ✓ ✓ ✓ ✓Routing by device ✓ ✓ ✓ ✓

23


Essential requirements fulfilled: browser tools

All offered device auto-detection and optimization for mobile, plus unwinding of grids

Only 2/7 products met all the key capabilities

6/7 supported email and SMS invitations/reminders

5/7 supported save and resume

Capturing of the agent string Most will record device data at a survey level;

Only 4/7 record the device at question level or allow conditional routing on the device type

24


Mobile app solutions

Analyzing the capabilities offered in the market

25


Mobile apps – capabilities offered by all vendors

Capability

Capture location data, with respondent

permission✓

Capture and upload photos and video ✓

Support for Android, BlackBerry and

Apple iOS✓

26


Mobile Apps – download method supported


Techneos SODA

Askia mobile

Global-park

Simple 1-click from a web link

✓ ✓ ✓ ✓ ✓

From an integrated panel portal

✓ ✓ ✓ ✓ ✓

From app site or App Store

✓ ✓ ✓ ✓ ✓

27


Mobile Apps – capabilities where support varies


Techneos SODA

Askia mobile

Global-park

Web browser authoring and deployment 1 ✓ ✓ ✓ ✓

Device-specific preview ✓ 2 ✓

Diary survey mode ✓ ✓ ✓ ✓

Diary alerts (regulated) ✓ ✓ ✓

Online/offline ✓ ✓ ✓ ✓

QR/Bar code scanning ✓ ✓ 1Authoring via either Confirmit or iBM SPSS Dimensions design module2via 3rd party simulator

28


Essential requirements fulfilled: Apps

6/6 will capture location data, photos or video, all under respondent control

6/6 offered a range of one-click download options

5/6 would operate without a constant internet connection

4/6 offer diary support and the other 2 are developing it Diary reminders are only fully supported (with the option for

participants to disable them) by 4 of the 5.

QR/bar codes supported by 4 but with 2 more developing

29


Discussion

Conclusions

Recommendations

30


Observations

Some vendors have specialized in serving the browser or the app-based solution, but many cater for both

All the browser solutions we examined will integrate seamlessly with standard online surveys (auto-detecting the device)

Most but not all vendors support a wide range of devices iOS, Android and Blackberry universally supported

Symbian only offered by 5/7 browser and 5/6 app products

XHTML supported by just 5/7 browser tools and the old (and very restrictive) WML/WAP by only 2.

31


Observations

Models of best practice are still developing

Previewing and simulating how the survey will appear on different small-format devices is really important Only 2/7 browser tools and 2/6 apps did this,

However, two more in each category plan to support this soon

Software developers are not necessarily taking advantage of the capabilities that mobile devices offer

32


Mobile research is still at an early stage of development

Support still rudimentary

Only two of the solutions met all of the key capabilities

Wide variation in what is offered by different vendors

platforms supported, key features etc

But developing fast

Most of the vendors recorded key capabilities that they were planning to introduce soon

Mobile research is still at an early stage of development

Support still rudimentary

Only two of the solutions met all of the key capabilities

Wide variation in what is offered by different vendors

platforms supported, key features etc

But developing fast

Most of the vendors recorded key capabilities that they were planning to introduce soon

Conclusions 1 of 2


Browser-based tools may offer better coverage but lack the more advanced mobile-specific capabilities

All browser-based solutions examined are competent at rending surveys on full-screen and mobile devices

Developers seem not to be aware of the importance of testing for survey designers

The more advanced and distinctive features of Mobile Research are not well supported in mobile apps yet

Conclusions 2 of 2


Further work

Future work needs to focus on the effect of mobile devices on conventional online research

Survey takers increasingly using mobile devices to take online Typically around 4% and growing

For them, the survey experience frequently degenerates into chaos

Web survey providers in general not addressing mobile surveys – desktop/laptop participation is assumed

35

36

Tim Macer [email protected]

Questions and comments welcome

Making it fit: How survey technology proviers are responding to the challenges of handling web...

Documents

Transcript of Making it fit: How survey technology proviers are responding to the challenges of handling web...