Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral &...

137
Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton, CA Vice President, Research & Development, UAD., Inc Theoretical Models of Explanation

Transcript of Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral &...

Page 1: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Robert A. Leark, Ph.D.

Fellow, National Academy of Neuropsychology

Associate Professor, Behavioral & Social Sciences

Pacific Christian College, Fullerton, CA

Vice President, Research & Development, UAD., Inc

Theoretical Models of Explanation

Page 2: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Multiple models of explanation for ADHD

Two have emerged as primary theories Barkley & Gordon Brown

Attention & executive functioning is multifaceted: difficult to map

Page 3: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Recent Historical Models Attention is not a unitary construct Zubin (1995): attention conceptualized

as having multiple components or elements

Psychiatric models:attention is process that controls the flow of information processing

Page 4: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Recent Historical Models Psychiatric models: 3 components of

attention:selectivitycapacitysustained concentrationAll of these must be sufficient enough to

interfere with daily activities

Page 5: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Recent Historical Models Neuropsychologists typically

conceptualize attention as:selective processingawareness of stimuli

Page 6: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Recent Historical Models Neuropsychologists use attention to

refer to:initiation or focusing of attentionsustaining attention or vigilanceinhibiting response to irrelevant stimuli

(selective attention)shifting of attention

Page 7: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Riccio, Reynolds & Lowe (2001) summarize components of attention

Arousal/alertness• motor intention/initiation

Selective Attention• focusing of attention (inhibiting/filtering)• divided attention• encoding, rehearsal & retrieval

Sustaining attention/concentrationShifting of attention

Page 8: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Broadbent (1973) - capacity to take in

information is limited, thus information not relevant needs to be filtered out. Information filtered out dependent upon stimulus characteristics (intensity, importance, novelty, etc.)

Page 9: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical 2nd model stresses arousal - here

optimal arousal (alertness) is necessary for effortful, organized function (Hebb, 1958)

Pribram (1975) - arousal is short-lived response to stimulus. Arousal is the general state of the individual that allows for & effects attentional processing

Page 10: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Mirsky (1987) proposed three factor

model for attentionfocusing of attentionsustaining of attentionshifting of attention

Page 11: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Mirsky model

selective attention: part of process of focusing attention (level of distractibility if deficient)

Sustained attention: ability to maintain that focus over time

Shifting of attention: necessary for adaptation & inhibition

Page 12: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Luria’s model

attention central to model2 attentional systems: reflexive & nonreflexivereflexive: orienting response/appears early in

developmentnonreflexive: result of social learning/develops

slowerlimbic system & frontal lobe mediate attention

Page 13: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Luria’s model

executive functions linked to mediating attention

executive functions:• self-direction• goal directedness• self-regulation• response selection• response inhibition

Page 14: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Mesulam (1981): model similar to Luria’s Model was specific to understanding

phenomenon of hemiattention or hemineglect as result of brain damage

Attentional processes: reticular system, limbic system, frontal cortex & posterior parietal cortex

Page 15: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Mesulam (1981) Subcortical influences from limbic

system, RAS & hypothalamus part of system matrix needed for control of attention

Frontal lobes influenced by & also influence the subcortical activity

Page 16: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Historical Summary: attention involves at least

two separate neural systemsactivation system: thought to be centered in

left hemisphere & involved in sequential/analytic operations

arousal: thought to be centered in right hemisphere & involved in parallel or holistic processing & maintenance of attention

Page 17: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Barkley & Gordon (1994,1997,1998,2001)

inattention emerges alongside a general pattern of impulsiveness & hyperactivity

deficits in self-control lead to secondary impairments in four executive functions

Page 18: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Barkley & Gordon (1994,1997,1998,2001)

Nonverbal working memory - sensing to the self

verbal working memory - internalized speech

emotional/motivation self regulation - private emotion/motivation to the self

reconstruction or generativity - cover play & behavioral simulation to the self

Page 19: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Barkley & Gordon (1994,1997,1998,2001)

basal ganglia dopaminergic disinhibition key factor to etiology

Page 20: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Barkley & Gordon (2001)

ADHD is a longstanding, pervasive and chronically impairing consequence of poor inhibition and/or inattention

model is consistent with the DSM-Ivr criteria

symptoms occur prior to age 7

Page 21: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Brown (1996) etiology is on purely inattentive stresses there has been an over-focus

on disinhibition and an under appreciation of arousal, activation and working memory

onset of symptoms can occur after age 7

Page 22: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Brown ADHD criteria includes inattentive

individuals who are not impulsive “all inattention is ADD/ADHD” ADHD is a suitable diagnosis for a broad

range of symptoms Brown’s rating scale: BADDS - modeled

upon this theoretical approach

Page 23: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Brown - ADD/ADHD is still an executive dysfunction of five clusters organizing & activating to work sustaining attention & concentration sustaining energy & effort managing affective interference utilizing working memory & recall

Page 24: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Theoretical Models of Explanation

Key components of models

inattention is the king of all nonspecific symptoms (Gordon, 1995)

inattention can emerge as a feature from a variety of psychiatric & medical circumstances

Page 25: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

History - conception through current age early life predictors

poor or inability to establish early life routinesmotor hyperactivity at early age

ADHD is a diagnosis by exclusion:low APGARhypoxiacentral nervous system diseases

Page 26: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Issues in Clinical Care

Page 27: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

History ADHD is a diagnosis by exclusion:

head injury/loss of consciousnessmetabolic disordersseizure disordersapneaother medical conditionsOther psychiatric conditions

Page 28: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

History ADHD is a diagnosis by exclusion:

ADHD is diagnosed only when other disorders do not best account for the symptoms

symptoms may be same, etiology somewhat different (or unknown)

treatment may even be the same

Page 29: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

History Problems with overlapping co-morbidity

create need to be able to stick to DSM IV criteria: age 7 issue

May not be possible to determine if signs & symptoms might have been present (such as trauma-abuse cases) if such trauma had not occured

Page 30: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Diagnostic procedures

Behavioral rating scales Measure of sustained attention &

impulse control Medication follow-up

Page 31: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales Child-Behavior Checklist (CBCL)

Parent RatingTeacher RatingItem pure scales: no item overlap

Page 32: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales BASC (Reynolds & Kamphaus)

Ages 2 - 18Item pure scales: no item overlapeasy to administershorter: about 140 items

Page 33: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales BASC (Reynolds & Kamphaus)

2-6: parent/other ratings7-12: self rating

parent rating teacher rating student observation guide

Page 34: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales BASC (Reynolds & Kamphaus)

13-18: self parent

teacher student observation guide

Page 35: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales BASC (Reynolds & Kamphaus)

New: ADHD predictorderived from discriminant function analysis using best predictors

Page 36: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Behavior Rating Scales Parent Ratings generally show more

impairment for child than do Teacher Ratings

May want to use “blind” ratings from Teacher - where Teacher is unaware of use of medication

helpful with treatment follow up studies

Page 37: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care Issues

Treatment Issues Treatment consistent with theoretical

models for ADHD? NIMH Treatment Guidelines

Medication effective, data indicated medication alone more effective than

• Medication & behavioral treatment• Behavioral treatment alone• Other modalities

Page 38: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care Issues

Behavioral therapies Treatment goal: improve/increase

inhibition Treatment strategies must be consistent

with goalTreatment strategies must be incorporated

into family system• Often source of increase problems if family not

stable• Noncompliance by parents

Page 39: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care Issues

Newer treatment modalities Neurofeedback

Issues:standardization of treatmentLength of treatmentTreatment cessation: maintenance of gains

Page 40: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Stimulant medication is standard of care NIMH revenue of ADHD studies

suggested thatStimulant medication alone better

than stimulant medication and behavioral therapy, behavioral therapy alone or placebo.

Page 41: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Medications

methylphenidate hydrochloride• Ritalin• Sustained Release• Concerta

Amphetamines• Adderall• Dexedrine

Page 42: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Medication Issues

kg/mg - is this an appropriate method for titration?

• Titration to cognitive measures produces an overall lower mean dosage than for behavioral measures

b.i.d. or t.i.d. • Dosage?• Time of day?

Page 43: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Behavioral Treatment

home and classroom based intervention strategies

requires cooperation of parents & teachers

effective - but best when used with medication

Page 44: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Family Therapies

Family system with behavioral interventions for child

Does require intact family system

Page 45: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Stimulant medication is standard of care NIMH revenue of ADHD studies

suggested thatStimulant medication alone better

than stimulant medication and behavioral therapy, behavioral therapy alone or placebo.

Page 46: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care Issues

Summary: treatment goals and plans need to be consistent with theoretical models of ADHD

Medication: ritalin, adderall, others

Page 47: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care Issues

Summary: treatment goals and plans need to be consistent with theoretical models of ADHD

Medication: ritalin, adderall, others

Page 48: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Continuous performance tests

Page 49: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Grew out of need to provide for a measurement of attention and impulse control

Wanted actual measurement not behavioral attributes

Page 50: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Advances in electronics provided format Historically, measures of sustained

attention are intrical to the history of psychology

Study cited as the basis for the origin of cpts is: Rosvold, Mirsky, Sarason, Bransome & Beck (1956). A continuous performance test of brain damage. Journal of Consulting Psychology, 20, 3343-350.

Page 51: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Background & History

For the Rosvold et al study (1956) the purpose was to study vigilance.

The designed task was for a letter to appear one at a time using a fixed rate of presentation (ISI) at 920 ms.

Press the lever whenever the letter x appeared

Page 52: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Background & History

The subject also had another task - to inhibit responding when any other letter appeared.

Task became known as the X type cptRosvold et al (1956) also reported use

of a second type cpt: the AX-typeFor this task, the subject was to press

the lever if a letter A preceded the letter X

Page 53: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Continuous Performance Tests

Still needed to inhibit actionAuthors found the task to adequately

classify 84.2% to 89.5% of younger subjects who had brain damage

Greater classification was for AX-type

Page 54: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Continuous Performance Tests

Since this study - have been literally hundreds of studies utilizing a cpt task of some sort- also report Riccio,Reynolds & Lowe (2001) over 400 articles using cpts

Riccio et al (2001) reported finding 162 research studies using some form of group comparison with children and some sort of cpt task

Page 55: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Continuous Performance Tests

Research studies may use a cpt designed only for that study lacking normative development increased difficulty with study

replicationEasy to program (if you find

programming easy)Many variations of design

Page 56: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Continuous Performance Tests

Cpt variations stimulus presentation interval of stimulus stimulus modality distraction modes adaptive cpts length of task target/nontarget ratio

Page 57: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Stimulus Presentation X- type (easier task) AX- type (more difficult task) XX-type Numeric (variation of X or AX type)

GDS uses numeric stimulus1 - 9 type task (number 1 followed by

number 9)

Page 58: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Interstimulus Interval (ISI) variations Rosvold et al (1956) used 920 ms some have used from 50 to 1500 ms

(Friedman, Vaughan & Erlenmeyer-Kimling (1981)

500 to 1500 ms (Schachar, Logan, Wachsmuth & Chajczyk, 1988)

some tasks maintain consistent ISI others use variable ISI within task

Page 59: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Other component related to ISI is that of stimulus onset asynchrony (SOA)

This refers to the onset of the stimulus followed by the onset of the next stimulus

i.d., stimulus may “linger” longer allowing task recognition

some cpts use variable SOA, others consistent SOA

Page 60: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

ISI - SOA increase ISI decrease SOA

shorter SOA may increase “mis-hits” shorter SOA may increase omissions

increase ISI increase SOAslower response times

Page 61: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Stimulus Modality (Visual/Auditory) Non-alphanumeric

Square within square (T.O.V.A.)Rabbit (in development)

Auditory stimulus presentation modelsauditory X or AX typesauditory numerictones (T.O.V.A.-A.)

Page 62: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Distraction these cpts use X or AX-type then add

another dimension: interference or distraction

goal is to increase level of difficulty distraction task varies by cpt

degraded or blurredvisual distractions common for visual X or AX

cptsauditory distractions

Page 63: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Adaptive cpts increase level of difficulty as success of

task accomplished and maintained

Page 64: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Length of task Bremer (1989) reported “mini-cpt”

3 minute task6 minute task available

T.O.V.A./T.O.V.A.-Alongest21.6 minutes

Page 65: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Variations of CPTs

Target/nontarget ratio refers to presentation of targets to

nontargets throughout task some use variable others consistent some use variable mixed with variable

ISI

Page 66: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Comments

Influences on cpt performance directions examiner presence anxiety, depression and the rest of

DSM-IV drugs and alcohol (including caffeine) environmental distractions

Page 67: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

4 major cpts have emerged within the marketplace

all report normative and standardization

Alphabetical order: Conners’ CPT (“The cpt”??) GDS IVA T.O.V.A./T.O.V.A.-A.

Page 68: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’ CPT Available from Multihealth Systems, Inc

(MHS)* www.mhs.com 800.456.3033

* may be available from other distributors such as PAR or WPS

Page 69: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’CPT Type: not x Modality: Visual Stimulus display 250 ms ISI varied 1000 to

4000 ms (varied

within block)

Page 70: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’ CPT Target Letter Length 14 minutes Nontargets letters Distraction none Target ratio not varied

Page 71: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’ CPT Block Timing yes Customized available Examiner presence ? Practice trials yes Standardized instructions yes

Page 72: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’ CPT Scoring correct hits omission/commission errors d-prime/beta reaction time reaction time standard deviation

Page 73: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Conners’CPT Scoring slope of standard error slope at ISI change slope of standard error at ISI change overall performance index

Page 74: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

GDS: Gordon Diagnostic System Available from: Gordon Systems, Inc.*

www.gsi.com 800.550.2343

* note: may be available from other distributors such as PAR, WPS

Page 75: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

GDS Type AX(numeric) Modality Visual Stimulus display 200 ms ISI 1000/2000 ms

(children adults/preschool)

Page 76: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

GDS Target number Length 9

minutes/6 for preschool Nontargets numbers Distraction yes Target ratio not varied

Page 77: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

GDS Block Timing yes Customized

available Examiner presence yes Practice trials yes

Page 78: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

GDS Scoring correct hits omission/commission errors reaction time target related error / random error

Page 79: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Intermediate Visual and Auditory CPT (IVA) also known as Integrated Visual & Auditory CPT Available from: BrainTrain*

www.braintrain-online.com 804.320.0105

* Note: May also be available from other distributors such as PAR, WPS

Page 80: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Type X Modality Visual & auditory in same

task Stimulus Display 167 auditory/500

visual ISI 1500 ms

Page 81: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Target

number Length 13 Nontargets numbers Distraction no? Target ratio varied

Page 82: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Block Timing yes Customized no Examiner presence yes Practice trials yes

Page 83: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Scoring response control quotient

(auditory,visual, full) attention quotient (auditory, visual, full) auditory & visual prudence scores vigilance consistency stamina

Page 84: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Scoring focus speed balance persistence fine motor/hyperactivity

Page 85: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

IVA Scoring sensoriomotor readiness comprehension

Page 86: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

Test of Variables of Attention (T.O.V.A.) & Test of Variables of Attention-Auditory (T.O.V.A.-A.) Available from: Universal Attention

Disorders, Inc. www.tovatest.com 800.729.2886 (800-PAY-ATTN)

*Note: Also available from other distributors such as PAR, WPS

Page 87: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

T.O.V.A./T.O.V.A.-A. Type: X Modality: Visual/Auditory Stimulus display 100 ms ISI 2000 ms

Page 88: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

T.O.V.A./T.O.V.A.-A. Target position of

square Length 21.6 mins Nontargets position of square Distraction no Target ratio varied

Page 89: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

T.O.V.A./T.O.V.A.-A. Block Timing yes Customized yes Examiner presences yes Practice trials yes

Page 90: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

T.O.V.A./T.O.V.A.-A. Scoring omission/commission errors response time response time variability d prime

Page 91: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

The Big 4

T.O.V.A./T.O.V.A.-A. Scoring multiple responses anticipatory Responses ADHD scale post commission error response time

Page 92: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Non-language based stimulusX-typeSquare within square stimulusSquare at top – targetSquare at bottom - nontarget

Page 93: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

T.O.V.A.-A. uses two tones: Middle c: non-target G above middle C: target

Consistent with paradigm: top is the target

Page 94: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Standardized instructions: to be given in language appropriate for subject (native)

Examiner must be present: standardization group did have examiner present

Prompt for subject to respond as quickly as possible when sees target

Page 95: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Separate standardization samplesOver 2500 subjects in T.O.V.A.-A.

Age 6 & above Ages 19-30

Over 2000 subjects in T.O.V.A. Age 4-5: 11.3 minute version One quarter of target

frequent/infrequent

Page 96: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

T.O.V.A. One year age increments ages 6 to 19 Data by gender Ages 20 & above: by decade Data by gender

Page 97: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Two conditions: target infrequent & target frequent

3.5:1 non-targets for every target (infrequent)

3.5:1 targets for every non-target: (frequent)

Stimuli presented in a fixed random model

Page 98: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Quarter 1 & 2: target infrequent Subject who is inattentive likely to miss

target Measure of attention Omission errors likely

Quarter 3 & 4: target frequent Subject who is impulsive likely to “mis-hit” Measure of impulse control Commission errors likely

Page 99: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Scores presented by quarters, halves & total for each variable

Scoring uses derived standard scores, 100 mean, 15 standard deviation

Higher scores reflect better performance, lower scores reflect poorer performance

Page 100: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

In addition: Z scores Percentiles for RT & RTV

Anticipatory errors Responses presented from 200 ms prior

to stimulus onset to 200 ms after onset

Page 101: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Multiple Responses: pressing button more than once

Post-Commission Response Time: following commission error, response time for next correct target identification is recorded

Page 102: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Multiple responses rare in standardization group Increased multiple responses decrease

validity of subject performanceError Analysis: examiner is able to

review all responses to all stimuli over duration of test

Page 103: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

ADHD score Based upon ROC discriminant function

analysis Best 3 predictors for placing subjects in

ADHD prediction group Uses subject z scores

Page 104: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

ADHD score Scores less than or equal to zero (0)

indicate subject more likely to be placed in ADHD group

Scores above zero (0) indicates subjects less likely to be placed in ADHD group

NOTE: RECALL THAT Z SCORES ARE USED TO DERIVE SCORES

Page 105: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

D Prime Measure of performance consistency

over duration of taskBeta: not found to be significant

between groups, thus is not reported

Page 106: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Construct validity Actual

Predicted Normal ADHDNormal 75% 25%ADHD 23% 77%

Leark, R.A., Dixon, D., Llorentes, A., Allen, M. (2000) Cross-validation & Performance Discriminant Abilities of the T.O.V.A. using DSM-IV criteria. Poster presentation at the 20th Annual Meeting of the National Academy of Neuropsychology. Orlando, FL.

Page 107: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Sensitive to malingering Increased errors across all 4 quarters,

both halves and total score for omission & commission

Decreased response time Increased variability of response time

Leark, R.A., Dixon, D., Hoffman, T. & Hunyh, D.(in press). Effects of Fake Bad performance on the T.O.V.A. Archives of Clinical Neuropsychology

Page 108: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Relationship to IQ Greenberg has reported need to adjust

T.O.V.A. scores for IQ HOWEVER – Research has indicated this

to be a false assumption

Page 109: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Chae (1999) T.O.V.A. not found to be significantly

correlated with VIQ/PIQ/FSIQ PIQ/FSIQ is moderately related to

Omission total scores ( .46 & .44) Picture Arrangement & Object Assembly

correlated at -.50 & -.54

Page 110: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Chae (1999) Freedom from Distractibility factor not

significantly correlated Processing Speed factor not significantly

correlated

Page 111: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Other studies have reported similar findings At best there is approximately a .50

correlation between FSIQ and T.O.V.A. scores

Third factor not significantly correlated with T.O.V.A. scores

IQ not factor in T.O.V.A. performance

Page 112: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Construct validity for T.O.V.A.-A ADHD (DSM-IV) to normal control children Diagnosis independent of T.O.V.A.-A.

performanceAll subjects correctly classified using z

scoresLeark, R.A., Golden, C.J., Escalande, A. & Allen, M. (2001) Initial

Dicriminant Abilities of the T.O.V.A.-A. Poster paper presented at the 21st Annual Meeting of the National Academy of Neuropsychology

Page 113: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Temporal Stability of T.O.V.A. Internal coefficients not appropriate for

timed tasks Temporal stability: reasonable time

interval90 minutes1 week

Page 114: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

90 Minute IntervalScale coefficient

Omission 0.80Commission 0.78RT 0.93RTV 0.77

Page 115: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

1 Week Interval Scale Coefficient Omission 0.86 Commission 0.74 RT 0.79 RTV 0.87

Page 116: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Sem

Scale 90 Minute 1 Week Omission 6.71 5.61 Commission 7.04 7.65 RT 3.97 6.87 RTV 7.19 5.41

Note: reflects T-scores

Page 117: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Relationship to behavioral rating scales Forbes (1998) reported that the T.O.V.A.

provided distinct information that added to increased diagnostic accuracy

Correlation studies have report significant but moderate correlations between behavioral measures and test variables

Page 118: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Forbes (1998) ACTers Hyper OM -.37 COM -.30 Oppos OM -.38 COM -.25 Attn OM -.25 COM -.16

Page 119: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Selden, Pospisil, Michael & Golden (2001)CBCL-TRF Attention Index

ADHD score .393TOVA-A COM .372

CPRS Hyperactivity ScaleTOVA OM .423

PIC-R Hyperactivity ScaleTOVA COM .325

Page 120: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Continuous Performance Test (CPT)

measure of sustained attention & vigilance

measure of impulse control long, boring measures

Page 121: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Test of Variables of Attention (Greenberg, 1992)

T.O.V.A. : non-language stimulus task computer based fixed two second interstimulus interval

(ISI) 21.6 minute long task

Page 122: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Nontarget Target

Page 123: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

two task paradigms: target infrequent & target frequent

a constant 3.5:1 ratio Target Infrequent: 3.5: 1 non-targets to

targets Target Frequent: 3.5:1 targets to non-

targets

Page 124: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Internally clocked Data summarized into quarters,

halves and total scoreQuarters 1 & 2 - target infrequent Quarters 3 & 4 - target frequentHalf 1 - target infrequentHalf 2 - target frequent

Page 125: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Extensive norm development: over 2300 subjects

Scaled by age and genderUses derived standard scores with

mean of 100, standard deviation of 15

z scores also provided

Page 126: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

T.O.V.A. Scales Omission - measure of

attention/inattention Commission - measure of impulse control Response Time - in milliseconds Response Time Variability - measure of

response consistency d’ (d prime) - signal detection measure

response consistency

Page 127: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Established construct and disciminant validity

Established reliability: 90 minute, 1 week, 8 week and 12 week intervals

Established sensitivity & specificity (80/20)

Page 128: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Semrud-Clikeman & Wical (1999) evaluated attentional difficulties in children

with complex partial seizures (CPS), CPS & ADHD, CPS without ADHD, and controls

used T.O.V.A. as measure of sustained attention & impulse control

Components of Attention in Children with Complex Partial Seizures with and without ADHD. Epilepsy, 40(2): 211-215.

Page 129: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Semrud-Clikeman & Wical (1999) Results: Found poorest performance on the

T.O.V.A. by the CPS/ADHD group. Difficulty in attention was noted for

children with epilepsy regardless of ADHD When methylphenidate was administered

to the ADHD groups - both improved on T.O.V.A. scores

Page 130: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Semrud-Clikeman & Wical (1999) Conclusions

Epilepsy may dispose children to attention problems that can significantly impair with learning

Improvement, as measured by improved T.O.V.A. measures was found for both ADHD groups when methylphenidate was administered

Page 131: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

T.O.V.A.

Mautner, Thakkar, Kluwe & Leark (in press)

NF1, NF1 with ADHD, ADHD & controls NF1 with ADHD & ADHD similar over 15% of the NF1 participants displayed

symptoms of ADHD Both the NF1 with ADHD and the ADHD

subjects had improved T.O.V.A. scores when methylphenidate was administered

Treatment of ADHD in NF1 Type 1. Developmental Medicine

Page 132: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Medications

methylphenidate hydrochloride• Ritalin• Sustained Release• Concerta

Amphetamines• Adderall• Dexedrine

Page 133: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Medication Issues

kg/mg - is this an appropriate method for titration?

• Titration to cognitive measures produces an overall lower mean dosage than for behavioral measures

b.i.d. or t.i.d. • Dosage?• Time of day?

Page 134: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Behavioral Treatment

home and classroom based intervention strategies

requires cooperation of parents & teachers

effective - but best when used with medication

Page 135: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Family Therapies

Family system with behavioral interventions for child

Does require intact family system

Page 136: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

Clinical Care

Treatment considerations Stimulant medication is standard of care NIMH revenue of ADHD studies

suggested thatStimulant medication alone better

than stimulant medication and behavioral therapy, behavioral therapy alone or placebo.

Page 137: Robert A. Leark, Ph.D. Fellow, National Academy of Neuropsychology Associate Professor, Behavioral & Social Sciences Pacific Christian College, Fullerton,

References