Ecological Validity of Ecological Validity of Neurocognition and Community Neurocognition and Community

FunctioningFunctioning

Debbie Gioia, Ph.D.Debbie Gioia, Ph.D.University of Maryland, BaltimoreUniversity of Maryland, Baltimore

School of Social WorkSchool of Social WorkResearch Lunch-Time Seminar SeriesResearch Lunch-Time Seminar Series

October 23, 2008October 23, 2008

dgioia@ssw.umaryland.edudgioia@ssw.umaryland.edu

Interest in neuropsychologyInterest in neuropsychology

• Clinical social worker on NIMH longitudinal Clinical social worker on NIMH longitudinal study at UCLA (18 years)study at UCLA (18 years)

• Funded R03 (2002-2004)Funded R03 (2002-2004)• The neuropsych puzzle of schizophreniaThe neuropsych puzzle of schizophrenia• Reduced cognitive capacity is one of the core Reduced cognitive capacity is one of the core

features of schizophrenia and independent of features of schizophrenia and independent of clinical symptomsclinical symptoms

• Unraveling the brain conditions that account for Unraveling the brain conditions that account for cognitive declinecognitive decline

Core social work constructsCore social work constructs

• Biopsychosocial(spiritual)Biopsychosocial(spiritual)

• Person-in-environmentPerson-in-environment

• Services for changeServices for change

These constructs of our profession provide These constructs of our profession provide legitimacy for me to enter an area of science legitimacy for me to enter an area of science that has not traditionally been populated by that has not traditionally been populated by social workerssocial workers

Schizophrenia researchSchizophrenia research

• Heterogeneity of the disorderHeterogeneity of the disorder• Tremendous individual variation in response to evidence-Tremendous individual variation in response to evidence-

based interventions (e.g., social skills training, ACT, based interventions (e.g., social skills training, ACT, supported employment)supported employment)

• Persons with schizophrenia perform typically 2 standard Persons with schizophrenia perform typically 2 standard deviations below the norm on neurocognitive testsdeviations below the norm on neurocognitive tests

• People with schizophrenia have the poorest functional People with schizophrenia have the poorest functional outcomes (e.g. work, social, independent living) of all the outcomes (e.g. work, social, independent living) of all the serious mental disorders serious mental disorders

• There is no effective treatment for cognitive deficitsThere is no effective treatment for cognitive deficits• Neurocognition and social cognition scores as predictors Neurocognition and social cognition scores as predictors

of psychosocial functioning (Brekke et al., 2005)of psychosocial functioning (Brekke et al., 2005)

Social work involvement with other Social work involvement with other populations with cognitive risk factorspopulations with cognitive risk factors

• Aging populations (all of us Aging populations (all of us ))• Alzheimer’s and mild cognitive impairmentAlzheimer’s and mild cognitive impairment• Substance usersSubstance users• Chronic physical conditions – e.g. cardiac, diabetes, Chronic physical conditions – e.g. cardiac, diabetes,

pulmonary, strokepulmonary, stroke• Persons with ADHD (children and adults)Persons with ADHD (children and adults)• PovertyPoverty• Poor nutritionPoor nutrition• Traumatic brain injuryTraumatic brain injury• DepressionDepression• Living with stress (all of us Living with stress (all of us ))

Components of MemoryComponents of Memory

EncodingEncoding (getting the information in)(getting the information in)

ConsolidationConsolidation(transferring information into long-term store)(transferring information into long-term store)

RetrievalRetrieval(getting the information out)(getting the information out)

Intact cognitive functioning defines who we areIntact cognitive functioning defines who we are

CognitionCognition

• From the Latin ‘cognoscere’: to come to knowFrom the Latin ‘cognoscere’: to come to know

• Cognition is the process of being aware, knowing, Cognition is the process of being aware, knowing, thinking, learning, reasoning, remembering, thinking, learning, reasoning, remembering, planning and judging.planning and judging.

• It can be closely associated with It can be closely associated with

emotionemotion

Measuring cognitionMeasuring cognition

Verbal LearningVerbal Learning: Ability to acquire, store, and retrieve verbal information for : Ability to acquire, store, and retrieve verbal information for more than a few minutesmore than a few minutes

Example:Example:

California Verbal California Verbal

Learning TestLearning Test

drill chivesplums tangerinesvest chiselparsley jacketgrapes nutmegpaprika apricotssweater plierswrench slacks

Why is verbal learning important?Why is verbal learning important?

• Helps to assess what an individual can Helps to assess what an individual can remember in a rehabilitation environment, remember in a rehabilitation environment, in a doctor’s office, or other learning in a doctor’s office, or other learning environment (following instructions)environment (following instructions)

• Helps individuals navigate their homes and Helps individuals navigate their homes and communities safelycommunities safely

• Enables individuals to incorporate adviceEnables individuals to incorporate advice

Immediate/Working memoryImmediate/Working memory

• Definition:Definition: Ability to hold information “online” in a Ability to hold information “online” in a temporary store or to manipulate the informationtemporary store or to manipulate the information

Example:Example:

Letter NumberLetter Number

Sequencing TestSequencing TestK3B4 _ _ _ _

R8C3G5 _ _ _ _ _ _

Why is working memory important?Why is working memory important?

• Remembering a phone number/addressRemembering a phone number/address

• Social conversation (storing information Social conversation (storing information about another person and maintaining your about another person and maintaining your side of the conversation)side of the conversation)

• Switching between different tasksSwitching between different tasks

Sustained attentionSustained attention

• DefinitionDefinition: Ability to respond to targets, not respond to : Ability to respond to targets, not respond to non-targets over a period of timenon-targets over a period of time

• Also called vigilanceAlso called vigilance• Involves cognitive and ocular motor processesInvolves cognitive and ocular motor processes• Usually a computerized test – individual asked to press Usually a computerized test – individual asked to press

whenever s/he sees a zero and ‘hold’ when another number whenever s/he sees a zero and ‘hold’ when another number appears appears

• Visual attention is critical for assessing and processing Visual attention is critical for assessing and processing motion information – moving cars, etcmotion information – moving cars, etc

• Example:Example: Continuous performance test (CPT) Continuous performance test (CPT) Startle Blink testStartle Blink test

The amplitude of the eye blink is a measure of how much you The amplitude of the eye blink is a measure of how much you startlestartle

Individuals with schizophrenia also startle to a non-startling Individuals with schizophrenia also startle to a non-startling quiet tone – failure of gatingquiet tone – failure of gating

Why is sustained attention Why is sustained attention important?important?

• In any social interaction there is a need to In any social interaction there is a need to maintain what is relevant and what is notmaintain what is relevant and what is not

• With ADHD and schizophrenia information With ADHD and schizophrenia information floods the individual and they have less floods the individual and they have less discretion in sorting out what is most discretion in sorting out what is most important for their attentionimportant for their attention

Executive functioningExecutive functioning

Definition:Definition: The cognitive system that is the manager of all cognitive The cognitive system that is the manager of all cognitive processes and heavily involved in our ability to handle new processes and heavily involved in our ability to handle new information and situations (prefrontal cortex)information and situations (prefrontal cortex)

Has been measured since 1950’s mostly with the Has been measured since 1950’s mostly with the Wisconsin Card Wisconsin Card Sorting Test-Sorting Test- persons with schizophrenia have significant difficulty persons with schizophrenia have significant difficulty (large effect size) in these measures compared to other psychiatric (large effect size) in these measures compared to other psychiatric patients (-1.45 compared to -0.40)patients (-1.45 compared to -0.40)

Why is executive functioning Why is executive functioning important?important?

• Helps us to correct errorsHelps us to correct errors

• Helps us to plan and structure our timeHelps us to plan and structure our time

• Helps us understand dangerous situations Helps us understand dangerous situations and make plans to avoid or reduce the and make plans to avoid or reduce the dangerdanger

• Helps us to moderate our impulses and Helps us to moderate our impulses and resist temptationsresist temptations

Why do we need to go to the next Why do we need to go to the next step?step?

• The relationship between neuropsychological (NP) The relationship between neuropsychological (NP) test scores and community functioning is strong test scores and community functioning is strong but do we really know what is being measured?but do we really know what is being measured?

• Low scores on NP tests predict poor outcome in Low scores on NP tests predict poor outcome in schizophrenia more strongly than psychotic schizophrenia more strongly than psychotic symptomssymptoms

• Developing functionally meaningful methods to Developing functionally meaningful methods to accurately capture and focus on community accurately capture and focus on community functioning outcomes that are correlated with NP functioning outcomes that are correlated with NP testing is just beginningtesting is just beginning

Ecological ValidityEcological Validity

• is defined as “the functional and predictive is defined as “the functional and predictive relationship between an individual’s performance relationship between an individual’s performance on a chosen set of neuropsychological tests and on a chosen set of neuropsychological tests and the individual’s behavior in a variety of the individual’s behavior in a variety of real world real world settingssettings”. ”.

• is related to external validity but differentis related to external validity but different• is related to the inferences drawn from the testis related to the inferences drawn from the test The test can be valid but not ecologically validThe test can be valid but not ecologically valid• Walking speed after TBI study (example)Walking speed after TBI study (example)

Ecological validity review article Ecological validity review article (Chaytor & Schmitter-Edgecombe, 2003)(Chaytor & Schmitter-Edgecombe, 2003)

• Studies reviewed – populations – closed Studies reviewed – populations – closed head injury, chronic medical conditions, head injury, chronic medical conditions, schizophreniaschizophrenia

• Concern:Concern: NP tests developed to provide NP tests developed to provide information about diagnosis are being used information about diagnosis are being used to answer questions about everyday to answer questions about everyday functioning with very little empirical functioning with very little empirical evidence to support this practiceevidence to support this practice

Two properties of ecological validityTwo properties of ecological validity

• VersimilitudeVersimilitudeThe degree to which the cognitive demands of a test resemble the The degree to which the cognitive demands of a test resemble the

cognitive demands of everyday lifecognitive demands of everyday life

• VerdicalityVerdicalityThe degree to which existing tests are empirically related to measures The degree to which existing tests are empirically related to measures

of everyday functioning such as clinician ratings, employment of everyday functioning such as clinician ratings, employment status etcstatus etc

Ecological questions are focused on what the client Ecological questions are focused on what the client actually does in the real world (messier) and usually not actually does in the real world (messier) and usually not the best performance state that is achieved in a lab (ideal)the best performance state that is achieved in a lab (ideal)

Trends from the reviewTrends from the review

• Most frequently used outcomes in studies – (1) return to Most frequently used outcomes in studies – (1) return to work and (2) IADLs obtained through clinician ratings and work and (2) IADLs obtained through clinician ratings and self-report – this review looked at everyday cognitive self-report – this review looked at everyday cognitive skillsskills

• Studies varied in choice of NP tests & outcomesStudies varied in choice of NP tests & outcomes

• Tests of memory related to everyday memory abilityTests of memory related to everyday memory ability

• Significant relationships between executive function tests Significant relationships between executive function tests and ratings by clinicians and others (not self-report)and ratings by clinicians and others (not self-report)

• The magnitude of the relationships between individual The magnitude of the relationships between individual tests and measures of outcome was weakly relatedtests and measures of outcome was weakly related

Sbordone & Long (1996)Sbordone & Long (1996)Ecological Validity of Neuropsych TestsEcological Validity of Neuropsych Tests

Vector approachVector approach – –

If all sources of If all sources of

information point information point

in the same direction, in the same direction,

the assessment is the assessment is

likely to be likely to be

ecologically valid ecologically valid

Findings from my R03Findings from my R03

• Gioia (in press); Gioia & Brekke (in press); Gioia (2006)Gioia (in press); Gioia & Brekke (in press); Gioia (2006)1.1. We found support for theWe found support for the ecological validity of NP test scores in that they were ecological validity of NP test scores in that they were

strongly associated with the character of community behavior in predictable and strongly associated with the character of community behavior in predictable and consistent ways for this sample of individuals with schizophrenia.consistent ways for this sample of individuals with schizophrenia.

2.2. There was some tendency for those with low NP functioning to have higher positive There was some tendency for those with low NP functioning to have higher positive symptoms.symptoms.

3.3. Self-esteem scores were lowest in those with low NP functioning, but satisfaction Self-esteem scores were lowest in those with low NP functioning, but satisfaction with life showed no difference between the two groups. with life showed no difference between the two groups.

4.4. The high NP group also showed deficitsThe high NP group also showed deficits 5.5. This method of observation and coding yielded useful and rich data on the range and This method of observation and coding yielded useful and rich data on the range and

predominant style of community functioning. This method showed sensitivity to predominant style of community functioning. This method showed sensitivity to functional variation not found in the quantitative measurement.functional variation not found in the quantitative measurement.

6.6. Even though there was a predominant behavioral style or strategy evident for each Even though there was a predominant behavioral style or strategy evident for each individual, there was notable heterogeneity in the behavior of each individual.individual, there was notable heterogeneity in the behavior of each individual.

7.7. Despite a range of functioning levels across the individuals, all were ‘making it’ in Despite a range of functioning levels across the individuals, all were ‘making it’ in complex real world situations.complex real world situations.

My R01 study proposalMy R01 study proposal

• Project rationale (a psychosocial rehabilitation/recovery philosophy)Project rationale (a psychosocial rehabilitation/recovery philosophy)

There is a real need to measure and treat neurocognitive deficits in There is a real need to measure and treat neurocognitive deficits in schizophrenia in order to enable more individuals with the disorder to schizophrenia in order to enable more individuals with the disorder to achieve their personal recovery goals and function as a full participant achieve their personal recovery goals and function as a full participant in the life of their community. This proposal’s novel use of mixed in the life of their community. This proposal’s novel use of mixed methods to assess ecological validity of NP tests with direct behavioral methods to assess ecological validity of NP tests with direct behavioral observation and use of co-primary measures will provide valuable observation and use of co-primary measures will provide valuable insights for the field in understanding the real world performance insights for the field in understanding the real world performance of individuals with schizophrenia; developing targeted interventions of individuals with schizophrenia; developing targeted interventions for cognitive deficits; documenting the resilience of individuals who for cognitive deficits; documenting the resilience of individuals who negotiate life’s tasks despite cognitive deficits; and reducing the public negotiate life’s tasks despite cognitive deficits; and reducing the public health risk. health risk.

DefinitionsDefinitions

• Direct behavioral observationsDirect behavioral observations

• Co-primary or (proxy) measures of Co-primary or (proxy) measures of cognitive functioning cognitive functioning • TABSTABS (Test of Adaptive Behavior in (Test of Adaptive Behavior in

Schizophrenia – Dawn Velligan)Schizophrenia – Dawn Velligan)• UPSA UPSA (The University of California (The University of California

Performance Skills Assessment – Tom Performance Skills Assessment – Tom Patterson et al)Patterson et al)

Proposed Study MethodProposed Study Method

• Sample (N=80) (age 18-50)Sample (N=80) (age 18-50)• Two sites (Harbor City & Mosaic)Two sites (Harbor City & Mosaic)• Measures – symptom, QOL, self-efficacyMeasures – symptom, QOL, self-efficacy• Eight step ethnographic procedure (handout)Eight step ethnographic procedure (handout)• Main goal – to add critical information through Main goal – to add critical information through

community observation of daily tasks to improve community observation of daily tasks to improve measurement of community functioning and measurement of community functioning and eventually to develop community performance- eventually to develop community performance- based interventions.based interventions.

Critiques from the reviewersCritiques from the reviewers

1.1. Consistent with national initiativesConsistent with national initiativesMATRICSMATRICS (Measurement and Treatment Research to (Measurement and Treatment Research to Improve Cognition) and Improve Cognition) and TURNS TURNS (Treatment Units (Treatment Units for Research on Neurocognition)for Research on Neurocognition)

2.2. Distinguishing specific NP tests related to Distinguishing specific NP tests related to community functioning in schizophrenia community functioning in schizophrenia (learning potential and verbal learning)(learning potential and verbal learning)

3.3. Non-cognitive confounding factors Non-cognitive confounding factors 4.4. Extent to which the observers change the Extent to which the observers change the

nature of the interactionnature of the interaction

Medications Medications

Clinical Antipsychotic Trials of Intervention Effectiveness Clinical Antipsychotic Trials of Intervention Effectiveness (CATIE) study (NIH)(CATIE) study (NIH)

• Comparing the effectiveness of older neuroleptics and Comparing the effectiveness of older neuroleptics and novel neuroleptics (over 1400 clients)novel neuroleptics (over 1400 clients)

• Some cognitive improvement with clozapine, riperidone, Some cognitive improvement with clozapine, riperidone, quetiapine reported in some but not all studiesquetiapine reported in some but not all studies

• Nicotine agonistsNicotine agonists

• Glutamate blockers (normalize abnormalities in prefrontal Glutamate blockers (normalize abnormalities in prefrontal contex)contex)

Effect of NicotineEffect of Nicotine

• Studies have suggested improvement in Studies have suggested improvement in some areas of attention and memory with some areas of attention and memory with the use of nicotine (nicotinic the use of nicotine (nicotinic neurotransmitters) (Dr. Gunvant Thaker)neurotransmitters) (Dr. Gunvant Thaker)

• Sensory gating improvesSensory gating improves• Swedish study of military recruits looked at Swedish study of military recruits looked at

early smoking (18-20 years) and lower rate early smoking (18-20 years) and lower rate of schizophrenia onset (protective factor)of schizophrenia onset (protective factor)

Cognitive RemediationCognitive Remediation

• Cognitive remediation is an array of Cognitive remediation is an array of behavioral interventions which use behavioral interventions which use computer programs and other targeted computer programs and other targeted learning tools to enable individuals to learning tools to enable individuals to retrieve some of their lost skills by teaching retrieve some of their lost skills by teaching a variety of compensatory strategies a variety of compensatory strategies

FutureFuture

• Re-submit R01 in March to NIMH & Re-submit R01 in March to NIMH & NARSAD with additional consult from Dr. NARSAD with additional consult from Dr. Jim Gold at MPRC and othersJim Gold at MPRC and others

• Continue learning in this area – reading, Continue learning in this area – reading, lectures, courseslectures, courses

• Possible text for social work studentsPossible text for social work students