Online Research Coming of age - Brownbag Presentation at Universitty of Pretoria SA
-
Upload
holger-luetters -
Category
Science
-
view
369 -
download
0
description
Transcript of Online Research Coming of age - Brownbag Presentation at Universitty of Pretoria SA
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Prof. Dr. Holger LüttersHochschule für Technik und Wirtschaft Berlin, Germany
Online Research Coming of AgeBrown-Bag presentation 18.09.2014
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Holger Lütters – a little older than the internet
My unplanned way into online research • First E-Mail sent 1994• First Webpage 1996• 1999 Own online survey created with
HTML • 2004 PhD on web based Analytic
Hierarchy Process • 2006 Interactive SniperScale• 2007 Panel study• 2008 International Multi-language
studies• 2010 Facebook for recruiting• 2011 Mobile only study• 2012 Avatar study • 2013 Gyroscale use • 2014 Mobile only Panel
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Problem with online market research:Lack of representativeness
Quelle: Fessel GfK Austria
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Online Market Africa: SA at 50% of Internet penetration leading on the continent, but still away from a representative popuation
http://www.internetworldstats.com/stats1.htm
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Methods of online research are not a substitute but a complement to the existing methods of market research
• Online market research has established itself as a serious approach (already 1st place in many countries)
• There are outstanding online solutions for many questions available
• For some topics online still is an inappropriate method due to the lack of representativeness of the population
• The time continues to work for the online market research idea:
• Expiration of classical structures of accessibility of people on the phone (5 phones in the home, multiple phone numbers, only mobile users, VoIP, etc.)
• Rising costs for traditional types of surveys
• Many online groups are already covered at 100% (students, teenagers, business surveys)
• Everything presented today will be in South Africa sooner or later. With our university cooperation it may be sooner and give you a competitive advantage over other institutions in your country and will help you compete with the international research community
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Fundamentals of online research
1
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
„Internet as an object“
Analysis of online users, online sites and online research
methods
„Internet as a method“
Collection of market and customer data with the help of
the Internet
Online Research: Internet as an object or a method?
Source: Zerr
Online-Research
Non-reactive data• User-Tracking• Visitor Analysis• Conversion Analysis• …
Reactive Data• Interviews• Focus Groups• Web-
Experiments• Delphi• …
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Online Market Research: Primary and secondary research with the help of Internet technologies
Example- Reports- Statistics- Databasesin - WWW- Intranet- Extranet- Social Media
ReactiveData
Non-reactiveData
Primary researchInternet as a tool
Secondary research (Desk)Internet as a source
Online-Market Research
Social Media Research
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Online-interview = location-independent, impersonal, self-administered computer-interview
Face-to-Face Telephone
impersonalpersonal
Computer-based surveys
Computer AssistedPersonal Interview
CAPI
Computer AssistedTelephone Interview
CATI
Computer AssistedWeb-Interview
CAWI
Many methodologically supported interviews are computer-assisted interviews ...... with the respective advantages and disadvantages of the specific interaction method
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
5 standard survey modes in 2014
Advantages of the different survey methods?Every method has its own purpose and target group
Effect of the chosen interview mode on results?Effect on cost, time, quality?
Personal Interview
Paper PencilSurvey
Telephone
Interview
Online
Interview
Mobile Interview
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Three steps to run a surveyonline
2
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Three key challenges arise after the decision for an online market research project
(Scientific) Modeling of
questionnaires
Development of a survey design
Programming
Distribution & Response
Management
Reach the defined
participants
ControllingScreen-out Quota Definition
Analysis & Reporting
Statistical analysis
Communication of results
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Evaluation of an online software for your own research project
Make sure that the software…
• Does what you need
• Has certain methodical features (Conjoint; PSM; AHP)
• Can be connected to a panel / CRM-System / Facebook
• Gives you total control over the entire study process (example: no advertising from third parties; no access from outside; no logos etc.)
• Creates appealing questionnaires
• Is accessible during and a certain time after the actual study (at least for more than 4 weeks / for free)
• Is exporting the data to SPSS/Excel
• Is easy to use (for surveymaker AND surveytaker)
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Online survey software should be online (SaaS)Some providers offer free access for non-commercial scientific research
http://www.gesis.org/dienstleistungen/methoden/beratungen/datenerhebung/online-umfragen/software-fuer-online-befragungen/#listings
Freeware:
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Creatingmotivatingquestionnaires
3
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Don’t worry about the duration. The first minute is the most important!
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
How do users perceive the interview experience?Leaving HTML-Standards towards bigger elements grows usability
HTML
AJAX
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Why taking over paper pencil standards onto the color screen?New forms are available and show very positive effects
Simple Radio Button
Simple Slider Design
Slider Design with Graphical Support
Flash Scale
Lütters, H. / Westphal, D. / Heublein, F. (2007): SniperScale: Graphical scaling in data collection and its effect on the response behavior of
participants in online studies; General Online Research Conference (GOR) 2007, Leipzig.
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
With growing bandwith more appealing questionnaires can be createdEffect: Higher willingness to participate
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Lütters/Westphal/Heublein (2007): “The future of graphical scaling is interactive”
Flash interaction with movement and sound
Non-interactive scales with AJAX technology
Interactive tests of the team based on Flash technology in 2007 Examples based on Ajax technology
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Audience adopted design creates higher response rates
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Adopt to the audience and the device used: Example of first iPad-only study
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
The progress bar: Never ending story of misunderstandings
Summary: • Has a strong effect (any kind of
undesired effect is negative)• Makes people thing about the
duration more than they should• Is only motivating when it is faked
(53% after 2 questions)• Faking vs. AAPOR/ESOMAR code of
ethics• People believe that it is a necessary
part of a questionnaire. It’s not!• Avoid the progress bar!
Try the count-down…
http://conftool.gor.de/conftool11/index.php?page=downloadPaper&filename=Callegaro-hould_we_use_the_progress_bar_in_online_surveys_A_meta-analysis-151.pdf&form_id=151&form_index=&form_version=final&CTSID_GOR11=AgPbzZ3ygg6WVMGwtHF8Y0
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Mobile research about to break through.Device Agnostic Research is the way to go on the long term
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
The questionnaire designed often looks different to interviewee than expected by the designer
Source: pangea labs; Dillman 2000: 360
OnlineQuestionn
aireDesigner
Designer‘s Hardware
& Software
Questionnaire
as seen by Designer
Questionnaire
as seen by Designer
Respondent‘s
Hardware &
Software
Online
Respondent
Questionnaire
as seen by
Respondent
Questionnaire
as seen by
Respondent
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Inviting people to online questionnaires
4
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Where will the participants be coming from?Why should they participate in such a survey?
• Employees• Customers• Company Website• Other Website• Convenient Sample
(Ponzi scheme )• Riversampling• Panelrecruiting• Offline•…
1. Topic2. Incentives
(monetary / non-monetary)3. Offering Results4. Design5. Anonymity6. …
• Open vs.. closed• Quota features• Plausibility check• Consistency Check• …
Controlling
of sources and quality of answers
Motivation
(to fully complete)
Sources of respondents
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Motivation to complete starts even with the invitation approachInfluence on opening rate and click rate
Sarah? Isabel?
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
SPAM as a central problem of email invitationsHTML vs. Plain Text:
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Social Media Recruiting is an opportunity which needs some extra care
0
20
40
60
80
100
120
140
160
An
za
hl
de
r S
tart
er/
Co
mp
lete
res
po
ns
es
Completes Social Media
Starter Social Media
14.2. 23.8. 5.10.
Daily average completion rate:
80%
First reaction to blogpost on web
First Invitation
as post on
adidas Running
Reminder
(Second Invitation)
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Combining offline and online with incentivesThe GAP invites to the customer survey on the receipt.
http://outlet-feedback.gap.com/
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Controlling of online interactions
5
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Controlling is important to see success and understand effects
datetotal no ofcontacts
quota full(interview terminated by questfox)
screened out(interview terminated by questfox)
No ofinterviewspossible Completed n= Completed %
26.08.2013 41 0 1 40 40 100 %
27.08.2013 175 0 3 172 159 92,4 %
28.08.2013 235 103 1 131 103 78,6 %
29.08.2013 1041 639 14 388 166 42,8 %
30.08.2013 1280 976 26 278 194 69,8 %
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Quality is insured by monitoring
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Controlling each question tells about problems of the questionnaire
question pageresponsesto question page
surveycompleted
Q1: ### Steuerung der Befragung 2772 (99,9% of total) 662 (23,9%)
Q2: Screener (Geschlecht) 2726 (98,3% of total) 662 (24,3%)
Q3: Screener (Alter) 2613 (94,2% of total) 662 (25,3%)
Q4: Screener (Bildung) 2597 (93,6% of total) 662 (25,5%)
Q5: Screener (Bundesland graphisch) 2576 (92,9% of total) 662 (25,7%)
Q6: Screener (Stadt oder Land) 2557 (92,2% of total) 662 (25,9%)
Q7: Screener (Location) 2547 (91,8% of total) 662 (26%)
Q8: Erfahrung mit Befragungen der Marktforschung 2536 (91,4% of total) 662 (26,1%)
Q9: Screener (Teilnahmebereitschaft) 2528 (91,1% of total) 662 (26,2%)
Q10: Screener (Device) 2773 (100% of total) 662 (23,9%)
T11: Politik, Liebe, Drogen 719 (25,9% of total) 639 (88,9%)
Q12: ### P01 - Regierung weiterempfehlen 707 (25,5% of total) 641 (90,7%)
Q13: P02 - Wo stehen Sie politisch? 700 (25,2% of total) 644 (92%)
Q14: P02_Sonntagsfrage 695 (25,1% of total) 642 (92,4%)
Q15: ### S01 - Beziehungsstatus 689 (24,8% of total) 641 (93%)
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Quota control attempts to portray the planned number of representative cases and close the study in case of reached limits
Example quota monitoring during an ongoing study
Example quota achievement after completing a study
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
IP to location creates detailed reports for regions (without asking)
Result of a UK study recruited over a panel
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Buying the respondents‘ willingness to participate
6
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Example of a worst case scenario: Recruiting from company database2780 email invitations resulted in 27 completed interviews
PERSONAL INVITATION INFO @ INVITATION
Source: B2B research project in EU research
Response to an international survey (6 languages) after one week
Σ=27
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Recruitment: Make or Buy?Recruiting for the same study with a commercial panel
Σ=638
Connection to a commercial panel in the same period
638 full interviews from panel vs. 27 full interviews with company e-mail-addresses
Source: B2B research project in EU research
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Many service providers are established in the new market of recruitment of participants (so-called online panel or MROC)
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
The price of ONE participant in an online study is a calculation based on duration and incidence of the targeted criteria
Example from SmartResearch in Euro: 1 € = 14 SAR
Number of questions
Panel
Number of Interviews
Incidence
9500 SAR
Drivers of the price• Duration (calculate € 0,10 cent per
Minute) thus less than 15 rand for a complete 10 minutes interview
• Only complete interviews are paid• 2 Minutes sampling time allowed to
define screen-outs and quota rulesIncidence:• 50% Male / Female• Car Owner: 70%• Owner of a Volkswagen: 10%
(equal to market share)• Owner of a VW Polo 1%In words: you need to contact 100 people to get one polo driver into your study. And what if he/she drops out?
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
The willingness to participate is guaranteed through incentivesThis approach creates less topic involvement
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Future of sampling: AYTM All-in-One-Solutions per Click
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Google consumer surveys is a combination of survey making and sampling at the same time
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
panelfinder.com as a meta platform for all kinds of panels
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Set up minimum standards and monitor your panel.Good Panels have full control over the relation to the participants
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Summary: Panels are very functional and cause less organizational pain… but need to be paid
Hints:• Always ask for 3 quotes from different companies and ask for their
calculation of the INCIDENCE• Only negotiate on full interview• Start small (n=300) -> you can still buy more respondents later• A panel telling you “no problem” might not be your first choice
Suggestion: • Why not doing general research in the USA (cheapest) or Europe (widely
available)?• Why not building up the first university panel in South Africa?
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
A few tips for your online project: Serious fun®
• Select the online method only when it is appropriate (not for financial reasons)• Try on-line studies: you will learn a lot about yourself and your relationship with
employees and customers or other groups much quicker• Take your time: An online project requires a much time as a classic data
collection project. Perhaps a little more when doing it for the first time• Emphasize more on methodology than on technology• Take care of your population. They are interviewed more often than
needed. Reconsider the number of required interviews in every study• Take care of every single respondent. Rethink difficulty, design, duration for
each project. Find an average between fun and torture.• Think from the perspective of your participants: Try to take into account an
external point of view throughout your project. Is it really as easy as you think?• Remain critical: Do not just mindlessly accept approaches from the past or
from offline into this new world. Go into a critical review process after a project • Online is different. Online needs to be different! Take advantage of the
benefits of a new world of data collection opportunities• Be open to new ideas. The internet is faster than you are. Try to incorporate
some of the brilliant ideas into your research
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Newer forms of online research
used by HTW Berlin Marketing Research Team
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Eyetracking: Registering peoples looks
Eyetracking of a single person Heat Map as aggregation of views
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Face Detection technology from the Fraunhofer InstituteGfK EMO Scan is the commericalised version
Idea: Facial Scan Field of application
• Gender recognition• Estimate of age• Mood measurement• Recognition of people in
groups• Recording the changes in
mood on exposition to changing stimulus (ex. Video ad)
• Possible with simple technology
http://www.youtube.com/watch?v=Fs-ngMK09Vk
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Listening is the new asking (Poynter)Insights from social media without asking anyone anymore
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Interaction through device movement: The Gyroscale literally moves answers
The Gyroscale® is available in questfox®Source: www.pangealabs.com Gyroscale® and questfox® are trademarks of pangea labs GmbH Switzerland
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
The avatars are empowered by text-to-speech technology which allows them to read out loud written texts
Different people have very different relations to ....
Text for reading:
Text is read by Avatar:
R1
R2
R3
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Outlook into the future ofonline research
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Some trends from the US are online on ist way to Europe
http://issuu.com/researchshare/docs/grit-winter-2014
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
#newMR is discussing the future of online
http://www.mrweb.com/mrt/
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Greenbook as a good online source to start with
http://www.greenbook.org/
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
What happens to market research, if the boundaries are broadened?
“In the future many professions and sectors will go the way of steam engine makers and the man who used to walk in front of cars carrying a red flag.
This might include the definition of market research as a business sector.”
Discussion under http://krz.ch/vkib
Holger Lütters - Online Research Coming of Age - University of Pretoria September 2014
Thank you for your interest Feel free to contact me
Holger Lütters – HTW Berlin
Contact informationProf. Dr. Holger LüttersHochschule für Technik und Wirtschaft Berlin
[email protected] Twitter @luettersxing.com/profile/Holger_Luettersde.linkedin.com/in/luetters/