Measures of Subjective Probability

Measuring Subjective Probability in the Context of Medication RisksNiels Haase, Frank Renkewitz & Cornelia BetschUniversity of Erfurt (Germany)

Haase, Renkewitz & Betsch: Measuring Subjective Probability 221st Annual Conference18-20 June 2012 ETH Zurich

Measures of Subjective Probability

7-Point Rating Scale

11-Point Rating Scale

Visual Analog Scale



Frequency Format

Percent Format



• Differences between scales– Resolution– Verbal vs. numeric

• Evaluative criteria– Usability– Test-retest reliability– Subjective confidence in judgment– Behavior prediction


Research Question & Design

• How do these scales perform when objective probabilities have been learned?

• SensitivityIndividual correlation between objective and subjective probabilities

• Context DependencyDifferentiation between ranges of probability

Objective Probabilities

Subjective Probability Judgment

Objective Probabilities

etc.(4 Levels of Probability)


Hypotheses

• Rating scales’ disadvantage due to low resolution and vague category quantifiers

• Visual analog scale’s disadvantage due to difficulties in its use

• H1: Numeric scales show higher sensitivity than the visual analog scale and the rating scales.

• H2: Rating scales and the visual analog scale will be more context-dependent.


Independent Variables

• 5 Scale Formats (between)

• 2 Ranges of probability (between):2%, 5%, 10%, 20%

vs. 42%, 45%, 50%, 60%

• Mode of presentation (within):SequentialGraphical (control)


Sequential Presentation


Sequential Encoding: Sensitivity


Sequential Encoding: Sensitivity

Low Range of Probabilities: 2%-20% High Range of Probabilities: 42%-60%


Sequential Encoding: Context Dependency7-point rating scale Percent format

42%20% 42%20%


• All scales were highly sensitive

• Numeric scales were generally superior

• Rating and visual analog scales were context dependent

• Differences between scales are reduced when encoding is more error-prone

Summary: Sequential Encoding


Probability of 42%:

Graphical Presentation


Sequential Encoding vs. Graphical Encoding


Graphical Encoding: Sensitivity


Graphical Encoding: Sensitivity

Low Range of Probabilities: 2%-20% High Range of Probabilities: 42%-60%


Graphical Encoding: Context Dependency7-point rating scale Percent format

42%20% 42%20%


• Altogether higher sensitivity

• Differences between measures remain the same, numeric formats superior

• Reduced context dependency

• No effect of range, i.e. encoding difficulty

Summary: Graphical Encoding


Experiment 2: Effect of encoding error

• Encoding more error-prone– 5 outcomes within 1 sequence– Shorter inter-stimulus time– 3 repeated measurements

• 2 Scale formats (between)7-point rating scale vs. Percent format

• 2 Ranges of probability (within):10%, 20%, 30%, 40%, 50% vs. 50%, 60%, 70%, 80%, 90%


Experiment 2: Sensitivity

Small Probabilities: 10%-50%

Large Probabilities: 50%-90%


Experiment 2: Context Dependency7-point rating scale Percent format

10% 20% 30% 40% 50% 50% 60% 70% 80% 90%Objective Probabilities

10% 20% 30% 40% 50% 50% 60% 70% 80% 90%Objective Probabilities

Small Probabilities: 10%-50%

Large Probabilities: 50%-90%

50% 50% 50% 50%


• Different scale formats vary in sensitivity with numeric measures generally faring the best

• Sensitivity also varies as a function of error in infomation at the time of judgment

• Scale characteristics (e.g. resolution) become less important with increasing error

• Context dependency increases with increasing error

• In the realm of probabilities rating scales are not suitable for between-subjects research

Conclusions & Implications


Thank you.

Measures of Subjective Probability

Documents

Transcript of Measures of Subjective Probability