Applying the Wiener diffusion process as a psychometric measurement model
description
Transcript of Applying the Wiener diffusion process as a psychometric measurement model
![Page 1: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/1.jpg)
Applying the Wiener diffusion process as a psychometric measurement model
Joachim Vandekerckhove and Francis Tuerlinckx
Research Group Quantitative Psychology, K.U.Leuven
![Page 2: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/2.jpg)
Overview
• An example problem
• The diffusion model
• Cognitive psychometrics
• Random effects diffusion models
• Explanatory diffusion models
• Conclusions
![Page 3: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/3.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships
P Block Item RT Answer
1 Mammals
1 Mammals
1 Mammals
1 Mammals
… …
![Page 4: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/4.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships
P Block Item RT Answer
1 Mammals Dog 523 “Yes”
1 Mammals
1 Mammals
1 Mammals
… …
![Page 5: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/5.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships
P Block Item RT Answer
1 Mammals Dog 523 “Yes”
1 Mammals Cat 475 “Yes”
1 Mammals
1 Mammals
… …
![Page 6: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/6.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships
P Block Item RT Answer
1 Mammals Dog 523 “Yes”
1 Mammals Cat 475 “Yes”
1 Mammals Spider 657 “No”
1 Mammals
… …
![Page 7: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/7.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships
P Block Item RT Answer
1 Mammals Dog 523 “Yes”
1 Mammals Cat 475 “Yes”
1 Mammals Spider 657 “No”
1 Mammals Wombat 723 “No”
… … … … …
![Page 8: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/8.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships– Measure both RT and response
![Page 9: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/9.jpg)
An example problem
• Speeded category verification task– Participants evaluate item category
memberships– Measure both RT and response– Each participant evaluates each
item exactly once– Expectation: “Typical” members
(e.g., Dog, Cat) are easier
![Page 10: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/10.jpg)
An example problem
• The problem– Standard assumptions violated
• Bivariate data (RT and binary response) do not conform to the assumptions made by standard models (e.g., normality)
– Different sources of variability• RT and response are partly determined by both
participants (ability) and items (difficulty)
![Page 11: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/11.jpg)
An example problem
• The problem– Standard assumptions violated
• Typical problem in mathematical psychology• Approach: use process models
![Page 12: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/12.jpg)
An example problem
• The problem– Standard assumptions violated
• Typical problem in mathematical psychology• Approach: use process models
– Different sources of variability• Typical problem in psychometrics• Approach: hierarchical models (multilevel models;
mixed models; e.g., crossed random effects of persons and items)
![Page 13: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/13.jpg)
Diffusion model
• Wiener diffusion model– Process model for choice RT– Predicts RT and binary choice simultaneously– Principle: Accumulation of information
![Page 14: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/14.jpg)
0.0 0.125 0.250 0.375 0.500 0.625 0.750
Evi
denc
e
τ
, , , ,pij pij pij pij pij pijx t Wiener
(For persons p, conditions i, and trials j.)
time
![Page 15: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/15.jpg)
Diffusion model
• Many associated problems– Technical issues
• Parameter estimation / Model comparison
– Substantive issues• Difficult to combine information across participants
– Problem if many participants with few data each– Problem if items are presented only once (e.g., words)
• Unlikely that parameters are constant in time (i.e., unexplained variability)
• Almost completely descriptive – differences over persons/trials/conditions cannot be explained
![Page 16: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/16.jpg)
Cognitive psychometrics
• Use cognitive models as measurement model
• Try to explain differences– between trials, manipulations and persons– by regressing the parameters on covariates
![Page 17: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/17.jpg)
Cognitive psychometrics
• Most common measurement model: Gaussian– Normal linear model (linear regression,
ANOVA):
– But often not a realistic model– Unsuited for choice RT
20 1 2
,pi pi
pi i p
y N
x z
Indexes p for persons, i for conditions
![Page 18: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/18.jpg)
Cognitive psychometrics
• Common measurement model in psychometrics: Logistic– Two-parameter logistic model (item response
theory):
1
0 1 2
1 pi
pi pi
pi
pi i p
y Bernoulli
e
x z
Measurement level describes the data
Regression component explains differences
Transform the parameter(s) to a linear scale
![Page 19: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/19.jpg)
Adding random effects
• Not all data points come from the same distribution
• Differences between participants/items/… exist, but causes unknown
,
1
pij pij pij pij pij
pij
p y Meas p d
p
Θ
![Page 20: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/20.jpg)
Adding random effects
• Case of the diffusion model’s drift rate
2
2
,
, , , | ,
, , , ,
,
pij pij
pij pij pij pij pij pi pi pij
pij pij pij pij pij pij
pij pi pi
p x t
Wiener N d
x t Wiener
N
![Page 21: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/21.jpg)
2
,
,
,
pij p p
pij p p
pij pi p
U
U
N
Adding random effects
, , , ,pij pij pij pij pij pijx t Wiener Measurement level (Wiener process)
Trial-to-trial variability in bias
Trial-to-trial variability in nondecision time
Trial-to-trial variability in information uptake rate
• Ratcliff diffusion model
![Page 22: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/22.jpg)
Adding random effects
• Crossed random effects diffusion model
2
2 2
, , , ,
,
0, and ,
pij pij pij pij pij pij
pij pi p
pi p i
p i
x t Wiener
N
N N
![Page 23: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/23.jpg)
Adding random effects
• Addition of random effects– Allows for excess variability
• Due to item differences• Due to person differences
– Allows to build “levels of randomness”– Importantly, can be accomplished with the
diffusion model– Only feasible in a Bayesian statistical
framework
![Page 24: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/24.jpg)
Applying to data
• Crossed random effects diffusion model
2
2 2
0,
, ,
p
i T T i D D
N
N or N
item easiness (distractors)
item easiness (targets)
person aptitude
Pop. distr. of
0.21 0.11
0.37 0.12
0.04
D D
T T
Mean Stdev
![Page 25: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/25.jpg)
Explanatory modeling
• Previous models were descriptive– Didn’t use covariates– Mixed models merely quantify variability
• Use external factors as predictors to– analyze the data – explain the differences in parameter values
(i.e., reduce unexplained variance)
![Page 26: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/26.jpg)
Explanatory modeling• Variability in choice RT due to
– Inherent (stochastic) variability in sampling– Trial-to-trial differences– Participant effects– Participant’s group membership– Item effects– Item type– Combination of the above– …
![Page 27: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/27.jpg)
Explanatory modeling
• Use basic “building blocks” for modeling– Random/Fixed effects– Person/Item side– Hierarchical/Crossed– Use covariatesUse covariates
(continuous/categorical/binary)
![Page 28: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/28.jpg)
Explanatory modeling
, , , ,pij pij pij pij pij pijx t Wiener
2
21 1 2 2
20 0 1 1
,
,
,
pij pi p
pi p i
p p p
i i i
N
N z z
N x x
explaining variability in drift rate
![Page 29: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/29.jpg)
Applying to data
2
2
20 1 1
,
0,
Typicality ,
pij pi p
pi p i
p
i i
N
N
N
0
1
2
0.9698
0.0659
0.0673
0.7178R
![Page 30: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/30.jpg)
Conclusions
• Category verification data– Variance in person aptitude small (0.04)
relative to variance in item easiness (≈ 0.11)– Item easiness correlates with typicality
![Page 31: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/31.jpg)
Conclusions
• More results (not discussed)– Other parameters besides drift rate may be
analyzed• e.g., encoding time is negatively correlated with
word length (at ±7ms/letter)
– Results hold across semantic categories (not just for mammals)
![Page 32: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/32.jpg)
General conclusion
• Hierarchical diffusion models – combine a realistic process model for choice
and reaction time with random effects and explanatory covariates
– allow to analyze complex data sets in a statistically (and substantively) principled fashion with relative ease
![Page 33: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/33.jpg)
Future work
• Efficient software for fitting hierarchical diffusion models
• Model selection and evaluation methods
![Page 34: Applying the Wiener diffusion process as a psychometric measurement model](https://reader030.fdocuments.net/reader030/viewer/2022033020/568139d4550346895da18641/html5/thumbnails/34.jpg)
Thank you
• Questions, comments, suggestions welcome