Debriefing in online experiment
-
Upload
harryadin-mahardika -
Category
Education
-
view
154 -
download
0
description
Transcript of Debriefing in online experiment
![Page 1: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/1.jpg)
Debriefing in Online Experiment
Harryadin Mahardika, PhD
![Page 2: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/2.jpg)
Harryadin Mahardika• Pop Economist
• FEUI & laporsuap.com
• Research objective:– “to liberate and empower consumer...”
• Current research:– Consumer empowerment
– Consumer intervention/engineering
– Mobile advertising
• Contact:– [email protected] / [email protected]
– @HarrySastro
2
![Page 3: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/3.jpg)
Discussion Agenda
• Informed consent• Debriefing• Randomization• Participant incentives
![Page 4: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/4.jpg)
Risk of experiment on human subject
• Deception, manipulation, priming, scenario.– May harm participants’ mind.– Mental fatigue.
• Examples?
![Page 5: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/5.jpg)
Informed consent• Information about the experiment and its risk.
– As a basis for participants to decide after they understand what the research involves (risks and benefits ).
• Written consent vs unwritten consent.– Written:
• Investigators must typically obtain and document voluntary informed consent from research subjects.
– Unwritten:• button on an online form to indicate they have read and
understood the consent form.
![Page 6: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/6.jpg)
Informed consent in online experiment• Limited interaction with participants
– investigator often cannot tell whether a subject understood the informed consent statement.
• Online form:– Researchers can increase the likelihood that subjects are granting truly
informed consent by requiring feedback from subjects about their level of understanding,
• Example: – by requiring a “click to accept” for each element in an informed consent
statement or even administering short quizzes to establish that a subject understood.
• Reduce response rate:– Increase nonresponse to sensitive items (Singer, 1978)– Possibly produce biased data (Trice, 1987).
![Page 7: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/7.jpg)
Risk in online experiment• It exposes subjects to innocuous questions and benign
or transient experiences with little lasting impact. • In general, online experiments is no more risky than any
of their offline counterparts. • In some respects, they may be less risky:
– The reduced social pressure (Sproull & Kiesler, 1991) in online surveys or experiments makes it easier for subjects to quit whenever they feel discomfort.
– This freedom to withdraw is no trivial benefit, given the strong pressures to continue in face-to-face studies (e.g., Milgram, 1963) and even telephone calls.
![Page 8: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/8.jpg)
Risk in online experiment• Although risk in online settings is typically low,
the actual risk depends on the specifics of the study.
• For example:– Some questions in a survey or feedback from an
experiment may cause subjects to reflect on unpleasant experiences or to learn something unpleasant about themselves
– e.g., Nosek et al.’s, 2002b, research on automatic stereotyping.
![Page 9: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/9.jpg)
Risk in online experiment
• Experiments that deliberately manipulate a:– subject’s sense of self-worth, – reveal a lack of cognitive ability, – challenge deeply held beliefs or attitudes, or – disclose some other real or perceived
characteristic
..... may result in mental or emotional harm to some subjects.
![Page 10: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/10.jpg)
Debriefing
• American Psychological Association (2002) ethical guidelines call for debriefing subjects:– “Providing an explanation of the nature, results,
and conclusions of the research—as soon after their participation as practical.“
• If deception was involved:– Researcher needs to explain the value of the
research results and why deception was necessary.
![Page 11: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/11.jpg)
Debriefing in online experiment
• When conducting research online:– Researchers can post debriefing materials at a
Web site, – Provide debriefing materials to those who leave
before completing the research (Nosek, Banaji, & Greenwald, 2002a).
– For example, researchers can deliver debriefing material through a link to a “leave the study” button or through a pop-up window, which executes when a subject leaves a defined Web.
![Page 12: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/12.jpg)
Debriefing in online experiment
• Appropriate debriefing in online research may be difficult:– The absence of a researcher in the online setting
makes it difficult to assess a subject’s state.– Difficult to determine whether an individual has
been upset by an experimental procedure or understands feedback received.
![Page 13: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/13.jpg)
Extraneous Variables cont.
Selection Bias Improper assignment of test units to
treatment conditions[sampling error]
![Page 14: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/14.jpg)
Extraneous Variables cont.
MortalityLoss of test units while the experiment is in
progress[respondents selected no longer wish to participate]
![Page 15: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/15.jpg)
Controlling Extraneous VariablesRandomisation Randomly assigning test units to experimental groups by using random
numbers
Matching Comparing test units on a set of key background variables before assigning
them to the treatment
Statistical Control Measuring the extraneous variables and adjusting for their effects through
statistical analysis
Design Control Use of experiments designed to control specific extraneous variables
![Page 16: Debriefing in online experiment](https://reader036.fdocuments.net/reader036/viewer/2022082502/5563b11cd8b42ad83c8b465f/html5/thumbnails/16.jpg)
Case: Beauty is in the eye of the beholder
Scientists have found a link between drinking alcohol and perceptions of beauty
80 students were shown colour photographs of 120 male and female students and were asked to rate the aesthetic properties on a 7-point scale from high unattractive to highly attractive
Half the students had drunk up to four units of alcohol, the other half had no alcohol.
The students who had consumed alcohol rated the people in the photographs as more attractive than the student who did not consume alcohol.
Source: http://www.theage.com.au/articles/2002/09/091031115991721.html