NIH Toolbox - HealthMeasures€¦ · NIHTB Auditory Verbal Learning Test (Rey) 8+ Instrument ....
Transcript of NIH Toolbox - HealthMeasures€¦ · NIHTB Auditory Verbal Learning Test (Rey) 8+ Instrument ....
NIH ToolboxAssessment of Neurological and Behavioral Function
www.NIHToolbox.org
Richard Gershon, PI Cindy Nowinski, Scientific Director Julie Hook, NIH Toolbox Product Manager Maria Varela, Director of Technology And a cast of thousands!
The NIH Toolbox was developed under contract to the NIH Neuroscience Blueprint starting in 2006. The goal was to develop assessments of neural and behavioral health functioning for use in large cohort studies and clinical trials. The assessments could be used as a form of “common currency” across diverse study designs and populations and would maximize yield from large, expensive studies with minimal increment in subject burden and cost.
The NIH Toolbox was initially made available to grants funded by the American Recovery and Reinvestment Act of 2009. The measures were released to the general research community in 2012.
The iPad version of the NIH Toolbox was released in 2015. In 2018 alone, the iPad version was used to administer 600,000 NIH Toolbox assessments in English, and over 30,000 in Spanish.
Since 2012, more than 100 validation studies have evaluated the NIH Toolbox for use in numerous additional sub-populations including aging, cancer, TBI, and SCI.
ClinicalTrials.Gov lists the NIH Toolbox as being used in 165 clinical trials.
Results of research conducted using the NIH Toolbox have been published in 253 peer reviewed journal articles.
89%
7% 4%
909* TotalSubscriptions
893 NIH Toolbox
34 PROMIS68 NIH TB Spanish
*NIHTB Cebuano has 2 subscriptions; All data from AppFigures
TOTAL
997
s listed in order of total sales; Data from AppFigures
*Purchased in 21 countries
Downloaded in 58 countries
Categories#
Inquiries App Support cases (NIH Toolbox, En Espanol and PROMIS) 1127 Cognition access application/Cognition unlocks 1136
PROMIS, NIH Toolbox and NIH Toolbox en español: 14 releases in 2018
Implementation of participant age as alternative to DOB New versions of NIH Toolbox language instruments:
Picture Vocabulary and Oral Reading Implementation of Emotion Composite/Summary Scores Improvements to Assessment and Longitudinal Reports Integration with Customer Service Desk.com system Improvements for handling of large Assessment
Databases
Audit log of Participant and Assessment actions Deletion of Participants and Assessment data Rescoring when participant demographics change Database of App diagnostic data to support troubleshooting Additional administration data added to exports Adaptation of Apps to new iPad models’ screens Bug fixes and upgrades to adapt to IOS changes
Spanish – complete Mandarin – active Portugese - active Hebrew – pending Arabic – pending Swahili and Dholuo - active Interest in several other languages
Instrument Group Name # AssessmentsNIHTB Pattern Comparison Age 7+ Instrument 50,327 NIHTB List Sorting Working Memory Test Age 7+ Instrument 49,479 NIHTB Flanker Inhibitory Control and Attn Age 12+ Instrument 42,926 NIHTB Picture Vocabulary Test Age 3+ Instrument 42,078 NIHTB Dimensional Change Card Sort Test Age 12+ Instrument 35,801 NIHTB Picture Sequence Memory Age 8+ Form A Instrument 34,950 NIHTB Oral Reading Recognition Test Age 3+ Instrument 29,873 NIHTB Flanker Inhibitory Control and Attn Age 8-11 Instrument 19,079 NIHTB Dimensional Change Card Sort Test Age 8-11 Instrument 16,882 NIHTB Flanker Inhibitory Control and Attn Ages 3-7 Instrument 12,757 NIHTB Dimensional Change Card Sort Test Ages 3-7 Instrument 11,167
Instrument Group Name # Assessments
NIHTB Emotional Support FF Age 18+ Instrument 6,843 NIHTB Instrumental Support FF Age 18+ Instrument 6,650 NIHTB Oral Symbol Digit Test 8+ Instrument 6,557 NIHTB Auditory Verbal Learning Test (Rey) 8+ Instrument 6,439 NIHTB Perceived Stress FF Age 18+ Instrument 6,382 NIHTB Loneliness FF Age 18+ Instrument 6,161 NIHTB Self-Efficacy CAT Age 18+ Instrument 6,048 NIHTB 9-Hole Pegboard Dexterity Age 3+ Instrument 5,794 NIHTB Sadness CAT Age 18+ Instrument 5,784 NIHTB Fear-Affect CAT Age 18+ Instrument 5,708 NIHTB Grip Strength Age 3+ Identifier 5,658 NIHTB Positive Affect CAT Age 18+ Instrument 5,600
Instrument Group Name # AssessmentsNIHTB Perceived Hostility FF Age 18+ Instrument 5,588 NIHTB Friendship FF Age 18+ Instrument 5,516 NIHTB Meaning and Purpose CAT Age 18+ Instrument 5,475 NIHTB Anger-Affect CAT Age 18+ v1.1 Instrument 5,465 NIHTB Perceived Rejection FF Age 18+ Instrument 5,402 NIHTB General Life Satisfaction CAT Age 18+ Instrument 5,395 NIHTB Picture Sequence Memory Age 8+ Form B Instrument 5,332 NIHTB Anger-Physical Aggression FF Age 18+ Instrument 5,198 NIHTB Anger-Hostility FF Age 18+ Instrument 5,128 NIHTB Words-In-Noise Test Age 6+ Instrument 5,095 NIHTB Fear-Somatic Arousal FF Age 18+ Instrument 5,029
Instrument Group Name # AssessmentsNIHTB Fear-Somatic Arousal FF Age 18+ Instrument 5,029 NIH TB 4-Meter Walk Gait Speed Age 7+ Identifier 4,605 NIHTB Visual Acuity Test Age 8+ Instrument 4,494 Picture Sequence Memory Age 8+ Form C Instrument 4,033 NIHTB 2-Minute Walk Endurance Test Age 3+ Instrument 3,484 PROMIS CAT v1.0 - Depression Instrument 3,455 NIHTB Pain Intensity FF Age 18+ Instrument 2,680 NIHTB Odor Identification Age 10+ 2,635 NIH TB Standing Balance Test Ages 7+ Instrument 2,558
Instrument Group Name # Assessments
NIHTB Flanker Inhibitory Control and Attn Age 12+ Formal Spanish 2,566 NIHTB Visual Acuity Test Age 8+ Formal Spanish Instrument 2,366 NIHTB Pattern Comparison Age 7+ Formal Spanish Instrument 2,078 NIHTB Fear-Affect CAT Age 18+ Spanish Instrument 2,032 NIHTB Anger-Affect CAT Age 18+ v1.1 Spanish Instrument 2,027 NIHTB Sadness CAT Age 18+ Spanish Instrument 2,023 NIHTB Positive Affect CAT Age 18+ Spanish Instrument 2,001 NIHTB Dimensional Change Card Sort Test Age 8-11 Informal Spanish Instrument 1,816 NIHTB Words-In-Noise Test Age 6+ Formal Spanish Instrument 1,553 NIHTB List Sorting Working Memory Test Age 7+ Formal Guatemalan 1,198
Instrument Group Name # Assessments PROMIS CAT v1.2 - Physical Function Instrument 1,214 PROMIS CAT v1.1 - Pain Interference Instrument 1,065 PROMIS CAT v1.0 - Depression Instrument 758 PROMIS CAT v2.0 - Satisfaction Roles Activities Instrument 646 NIH Toolbox General Life Satisfaction CAT Age 18+ Instrument 441 Neuro-QoL CAT v1.0 – L Ex Function (Mobility) Instrument 411 PROMIS CAT v1.0 - Anxiety Instrument 329 PROMIS Bank v1.2-Upper Extremity Instrument 317 PROMIS Ped Bank v2.0-Pain Interference Instrument 275 PROMIS Bank v1.0-Self-Efficacy Manage Symptoms Instrument 268 PROMIS Bank v1.2-Mobility Instrument 221 PROMIS CAT v1.0 - Sleep Disturbance Instrument 208 Neuro-QoL CAT v1.0 - Stigma Instrument 208 PROMIS Ped Bank v2.0-UpperExtremity Instrument 208
International Neuropsychological Society (February, 2019) Hispanic Neuropsychological Society (February, 2019) American Congress of Rehabilitation Institute (April, 2019) Gatlinburg Conference for Intellectual Disability (April, 2019) HealthMeasures Users Conference (June, 2019) Alzheimer’s Association International Conference (July, 2019)
Presentations and Poster sessions
American Psychological Association (August, 2019) Exhibitor Booth, Poster sessions, and Div 40 Presentation
American Congress of Rehabilitation (November, 2019) Exhibitor Booth and Workshop
Social Media AccountsFaceBook = 147 FollowersLinkedIn = 57 FollowersTwitter = 390 Followers
NIH Toolbox publicly released in September 2012 on Web National norms (enhanced in 2015), well validated
Large, NIH-sponsored research projects desired more portable solution
iPad solution initially released for NIH use in 2014, public use through App Store in 2015
Some studies continued to use Web version, some initiated with iPad; others transitioned and have both types of data
Web Version iPad Version
Stand-alone laptop computer with separate 19” monitor for test presentation
Uses single screen for both test presentation and participant responses – 9.7”
Participant responses recorded via keyboard (reaction time tests) and/or mouse
Uses touchscreen interface for all participant responses
Internet connection required, data loaded from web
No internet connection required for testing; only for uploading data
After review, we concluded that no mode effects would be expected for Motor, Sensation or Emotion measures
Vision test distance shortened to 3 M, optotypes mathematically adjusted for distance, screen size and resolution
Others do not require interaction with iPad or involve response selection only (e.g., PROs)
Performance differences on some Cognition tests deemed possible (hypothesized not likely for language measures)
Mode of administration did not influence test scores on three NIHTB tests: Picture Vocabulary Reading List Sorting
Four tests showed differences between Web and iPad, requiring adjustments: Picture Sequence Memory DCCS Flanker Pattern Comparison
NIH ToolboxAssessment of Neurological and Behavioral Function
~11,000 children between the ages of 9 and 10 Scanned every 2 years longitudinally
NIH Toolbox Cognition A subset of emotion measures
Phenotypic measures Diagnostic interview Measures of mental health and substance use related behaviors
Imaging High resolution T1 and T2 diffusion scan minutes of resting state tasks in the scanner
1,200 young adults between the ages of 22 and 35 Many sets of DZ and MZ twins
Full NIH Toolbox other than balance and vision Full diagnostic interview Imaging High resolution T1 and T2 60 minute diffusion scan 60 minutes of resting state 7 different tasks in the scanner
1,340 individuals between the ages of 5 and 21 A subset scanned multiple times
Full NIH Toolbox other than balance and vision Full diagnostic interview Imaging High resolution T1 and T2 22 minute diffusion scan 40 minutes of resting state 3-4 different tasks in the scanner
1,340 individuals between the ages of 35 and 100 A subset scanned two times
Full NIH Toolbox other than balance and vision Full diagnostic interview Imaging High resolution T1 and T2 22 minute diffusion scan 40 minutes of resting state 3-4 different tasks in the scanner
ECHO is a 7-year (2016-23) NIH-funded large research consortia of 84 existing longitudinal cohort studies representing 40 states and Puerto Rico, n~50,000 children
ECHO aims to leverage diverse cohorts to address a broad range of early environmental exposure effects (physical & chemical, societal, psychosocial, behavioral, biological) on children’s health outcomes by standardizing new data collection and harmonizing existing measures The goal is to advance scientific understanding to inform
high-impact programs, policies, and practices that improve the health of children and adolescents
NIH Toolbox
Physical Motor Battery
Neuro-developmental
Perceived Stress, Early Childhood Cognition Battery, Cognition Battery, Pattern Comparison
SocialPositive Peer Interactions, Friendship, Loneliness, Emotional Support
NIH Toolbox
Neuro-developmental Perceived Stress, Cognition Battery
Matrix Reasoning Test, Ages 5-85 Nonverbal/visual reasoning task Undergoing IRB review
Concept Formation, Ages 2-5 Basic cognitive task that measures perceptual and semantic reasoning Test item layouts being specified
Speeded Matching, Age 3+ Processing speed test Pilot testing March 2019
iPad based eye-tracking task Social attention/cognition task for infants Currently exploring methods for adapting adult-oriented face recognition and
gaze detection algorithms for use with infants
Developed a bi-directional interface between REDCap instances and the NIH Toolbox App Participants and assessments from REDCap projects are downloaded
to the NIH Toolbox App. Scores and item data uploaded to the corresponding REDCap project from the NIH Toolbox App
Beta release to small group of ECHO cohorts to follow Collaborated with the REDCap consortium group at Vanderbilt for an
alpha version of their personal research app, MyCap, which can administer IRT-based NIH Toolbox instruments Now available for both iPhone and Android
Validate the NIHTB (English & Spanish) in ethnically/racially diverse adults ages 65-85 along the trajectory of cognitive aging and in cognitively normal individuals aged 86+.
Expand the NIHTB by adding innovative instruments to assess neurological functions associated with cognitive aging.
Facilitate use of the NIHTB + new measures in aging research by ensuring it is a readily available resource with robust user support.
Multi-center: 9 participating sites from across the United States Northwestern, Massachusetts General, University of Wisconsin, Mayo Clinic
Jacksonville, University of California San Diego, University of Pittsburgh, Emory University, Columbia University (NY)
Recruit and follow 1,260 patients over 2 years 29% (n=370) Spanish-speaking 24% (n=300) African American
Biomarkers and clinical data for validation of NIHTB sourced through the Uniform Data Set (UDS) and managed by the National Alzheimer's Coordinating Center
New NIHTB measures incorporated into HealthMeasures Obtain FDA approval for use of the NIHTB+ in clinical trials
The Face-Name (FNAME) Associative Memory Test to increase sensitivity to early cognitive changes
An iPad-based test of Near Visual Acuity, because it is critical for interpretation of NIHTB performance but not currently measured by the NIHTB
A Pure Tone Audiometry Test to identify hearing threshold levels of an individual, enabling determination of the degree, type and configuration of a hearing loss.
1. Create the MobileToolbox Library of iOS and Android smartphone based assessments.
2. Develop and maintain the Mobile Toolbox Research Platform, a software platform to support clinical researchers and software developers in the dynamic, customizable integration of Mobile Toolbox Library into research studies
3. Conduct psychometric activities to validate, norm and assess reliability of the MobileToolbox in both general population and various research samples to create general population reference norms
The Case for the use of the NIH Toolbox for Clinical Assessment
NIH Toolbox Cognition Battery (NIHTB-CB) has been “validated as a research test battery and not for clinical use, nor would it substitute for a comprehensive clinical neuropsychological examination of patients with neurobehavioral symptoms and disorders.
It has several potential applications in clinical research and in longitudinal, large-scale epidemiologic studies where there is a need for brief instruments that tap different cognitive constructs within a very large age range …”
Weintraub, Dikmen, Heaton, Tulsky, Zelazo, et al. 2014. P. 568
NIH Toolbox was developed using traditional test development techniques. The battery was developed with extensive item development stages, tryout,
validation, and standardization. The NIHTB has state-of-the-art normative scores. The NIHTB has demographically corrected normative data. All point to the potential to use the NIHTB-CB in clinical practice.
However, all of the original development work was with “healthy” individuals..
41 studies published that used the NIH Toolbox in a clinical population 10 studies were conducted among cancer patients, of which 9 were
breast cancer patients/survivors 12 studies were conducted in patients with Traumatic brain injury
(TBI), Spinal cord injury (SCI), and stroke 39 studies used cognition battery or subset of cognition battery, 3
studies used emotion battery, 2 studies used motor tests, and 1 study used sensation tests
Pop/Conditions
Group NIH-TB Measure Study design
Dependent and Outcome Variables
Journal
Breast cancerPatients
Northwestern(2,18)
Cognitive Battery Casecontrol
Hippocampal functional connectivity, subtle hippocampal deformities,episodic memory, and cognition
NeuroImage: Clinical
City of Hope National Medical Center (3,4,6)
Cognition Battery Cohort Brain gray matter density, subcortical gray matter structures, brain volume changes, and cognition.
Breast cancer research and treatment, Magnetic resonance imaging, Breast Cancer Research.
Harvard & U of Wisconsin (13)
Cognition Battery Cross-sectional
Sedentary behavior and objective cognitive functioning
Supportive Care in Cancer
Texas State U(17)
Cognition Battery Cross-sectional
Low Serum Carotenoids andcognitive dysfunction
Nutrients
U of Alabama at Birmingham(34)
Executive function
Cohort Processing speed training andexecutive function
Breast Cancer Research and Treatment
UC-San Diego(26)
Cognition Battery RCT Physical activity intervention and cognition
Cancer
Oropharyngeal Cancer Patients
Washington U in STL(7)
Cognition Battery Cohort Baseline subjective vs objective cognition
JAMA
Conditions Group NIH-TB
MeasureStudy design
Dependent and Outcome variables Journal
Traumatic brain injury (TBI)
Harvard (5) Cognition Battery
Cross-sectional
Depression and cognition The Clinical Neuropsychologist
Ohio State U(32) Cognition Battery
Cross-sectional
TBI and cognition American Journal of Speech-Language Pathology
TBI, Spinal cord injury (SCI), and stroke
Northwestern, U of Delaware, U of Michigan, Washington U,Rehabilitation Institute of Chicago, University of Illinois at Chicago, UC- San Diego, etc(19,20,25, 28,29, 30,33,36,39)
Cognition Battery
Cross-sectional
Validation and test accommodation of NIHTB-CB in patients with TBI,SCI, and stroke; Use NIHTB-CB to Identify cognitive impairment in TBI patients; cognition of SCI patients; racial differences in cognition among post-stroke patients; health literacy and cognition in TBI,SCI, and stroke patients .
Rehabilitation Psychology, Archives of Clinical Neuropsychology, Journal of the International Neuropsychological Society, HLRP: HealthLiteracy Research and Practice
Acquired brain injury
Northwestern, U of Delaware, U of Michigan, Washington U,UC- San Diego, etc (40)
Cognition Battery
Cross-sectional
Evaluated the NIHTB-CB factor structure in individuals with acquired brain injury vs general pop
Rehabilitation Psychology
Conditions Group NIH-TB MeasureStudy design
Dependent and Outcome Variables Journal
Epilepsies U of Alabama at Birmingham(8)
Flanker Cross-sectional
Cortical excitability, seizure control and cognitive performance
Epilepsy and Behavior
Cincinnati Children's Hospital(14)
DCCS, Flanker, LSWM Cross-sectional
Executive Function deficits pattern, epilepsy type, and measurement tools
Epilepsyand Behavior
Alzheimer's and Dementia
Temple, Cornell, & U of Cincinnati(11,41)
Cognition battery Cross-sectional
Validate NIH-TB for clinical assessment, propose a novel framework in evaluate and treating AD patients in clinic
Alzheimer’s & Dementia
Patients at Brain Cognitive Screening Clinic
Emory (12) PSM Cross-sectional
Clinical use of PSM and its Diagnostic Relationship to the Rey Auditory Verbal Learning Test
Archives of ClinicalNeuropsychology
Patients with diffuse glioma
U of Calgary in Canada(31)
Cognition Battery,Emotion Battery,motor tests (Grip Strength & Pegboard)
Cohort Use NIHTB to assess emotion, cognition, and motor at baseline and throughout treatment.
World Neurosurgery
PTSD & Chronic Musculoskeletal Pain
U of Arkansas(38) Flanker, LSWM, PSM, Emotion Battery
Quasi-experimental
Taichi intervention, pain, emotion, and cognition
Rehabilitation Psychology
Conditions Group NIH-TB
MeasureStudy design
Dependent and Outcome Variables Journal
Veterans with major depression
Oregon Health & Science U(16)
Adult Social Relationship Scales
Cross-sectional
Social connectednessand depression
Arthritis Care and Research
Schizophrenia U of Texas Health Science(9,21,35)
Cognition battery
Cross-sectional
C-reactive protein,S100B calcium binding protein, selectins, and cognitive function
Frontiers in Public Health
Binge-Eating Disorder Temple U(23) DCCS, Flanker, PCPS, LSWM, PSM
Cross-sectional
Executive functioning and Binge-Eating
International Journal of Eating Disorders
Cocaine Use Disorder Columbia U(24) Cognition battery
Case-control
Cocaine use and cognitive functioning
Journal of Clinical and Experimental Neuropsychology
Chronic Marijuana Users U of Hawaii (37) Cognition battery
Case-control
Microbiota, mitochondrial function, and cognition
Journal of NeuroimmunePharmacology
Conditions Group NIH-TB
MeasureStudy design
Dependent and Outcome Variables Journal
Pediatrics Preterm infants U of Minnesota(1)
PCPS Cohort Postnatal body composition, speed of processing, and blood pressure
Pediatric research
Boys with Dystrophinopathies(aged 5-17)
U of New York(10)
Flanker, DCCS, LSWM,PCPS
Cross-sectional
Executive skills and academic achievement
Journal of the International Neuropsychological Society
Children with Recurrent Acute Tonsillitis
Dartmouth Medical Center(27)
Odor Identification Test
Cohort Recurrent acute tonsillitisand glossopharyngeal taste
Otolaryngology -Head and Neck Surgery
Preschool children with sickle cell disease
UCollegeDublin(22)
Flanker Case-control
Sickle cell disease and selective attention
Journal of Pediatric Psychology
Adults Systemic lupuserythematosuspatients
Emory (15) PSM, LSWM,PCPS,Flanker, DCCS
Case-Control
Physical and cognitivefunction in patients with SLE
Arthritis Care and Research
Update the NIH Toolbox to fully maximize iPad Delivery (2019) Renorm all of the measures in the NIH Toolbox (2020) Release NIH Toolbox Version 3.0 with updated measures and new
norms (2021)