Understanding Concussion

Assessment and Evaluation

Philip Schatz, PhDSaint Joseph’s University, Philadelphia PA

pschatz@sju.edu

Why should we opt for computer-based assessment?

• Sensitive to RT, Processing Speed

• Randomized Trials: Improved reliability

• Ability to test entire team at once (benefit?)

• Better/Objective date for Athletic Trainers, Team Physicians

• Assistance with Return to play decisions (Schatz & Zillmer, 2003, Applied Neuropsych; Schatz & Browndyke, 2003, JHTR)

Why should we not opt for computer-based assessment?

• Perhaps timing is not millisecond-accurate

• Limited validation with np “standards”

• Little qualitative or “verbal” data

• Can be used by Athletic Trainers, Team Physicians in absence of Neuropsych

• May determine return to play decisions (Schatz & Zillmer, 2003, Applied Neuropsych)

Cross-Validation of Computer-Based Measures - Rationale(Schatz & Putz, Applied Neuropsych 2006)

• Limited “shared validation” of existing computer-based measures with standards

• SDMT correlates with:– ImPACT Processing Speed (r=.70) and Reaction Time

(r=-.60)– CRI Processing Speed (r=.60, r=.67)

• Trails correlates with:– CRI Response Speed (A: r=.73; B: r=.74)– CRI Processing Speed (B: r=.37)– CogSport Complex RT (B: r=.34)– CogSport Simple RT (B: r=.44)

Cross-Validation of Computer-Based Measures - Methodology

• 30 Normal Volunteers

• Computer-based: ImPACT, CRI, CogSport, d2, Trails A&B, Digit Symbol

• Paper-based: Trails A&B, Digit Symbol

• Administration: MWF, Individually– Grouping A: ImPACT, d2 Test of Attention (computerized)

– Grouping B: CRI, Trails A and B. and Digit Symbol (pencil and paper).

– Grouping C: CogSport, Trails A and B, and Digit Symbol (computerized).

– ABC, ACB, BAC BCA, CAB, CBA.

Cross-Validation Example: ImPACT CRT

Cross-Validation Example: CogSport CRT

Cross-Validation Example: CRI CRT

Cued Reaction Time:Press the spacebar as quickly as possible only when a white circle immediately follows the presentation of a black square.

Cross-Validation of Computer-Based Measures - SRT Results

Cross-Validation of Computer-Based Measures - CRT Results

Cross-Validation of Computer-Based Measures - Processing Speed Results

Cross-Validation of Computer-Based Measures - Memory Results

Cross-Validation of Computer-Based Measures - Implications

• These tests share some common variance on constructs such as processing speed and reaction time, but not within the domain of memory.

• The Processing Speed Indices/measures (ImPACT, HeadMinder, Trails B and Digit Symbol) correlated the most consistently.

• Baseline evaluations using one measure can not be used as a basis for post-concussion assessment using another measure.

Cross-Validation of Computer-Based Measures - Limitations

• Small sample size.

• Reliance on univariate correlations.

• Subjects assessed independently.

• No concussed subjects.

• Not comparing tests’ sensitivity to effects of concussion.

NEW MANAGEMENT STRATEGIESNEW MANAGEMENT STRATEGIESCIS Group, Vienna (2001), Prague (2004)CIS Group, Vienna (2001), Prague (2004)

Removal of symptomatic athletes from playRemoval of symptomatic athletes from play

Graduated return to playGraduated return to play

Neuropsychological testing recommendedNeuropsychological testing recommended

““One of the Cornerstones of ManagementOne of the Cornerstones of Management””

Aubry, Cantu, Dvorak, Graf-Baumann, Johnston, Kelly, Lovell,McCrory, Meeuwise, Schasmasch, 2001. Clinical J. Sports Med.

Testing Provides Testing Provides Unique InformationUnique Information

Concussed Athletes Often Concussed Athletes Often Deny SymptomsDeny Symptoms

Athletes May Athletes May Lack Awareness of SymptomsLack Awareness of Symptoms

WHY USE NEUROPSYCHOLOGICAL TESTING?WHY USE NEUROPSYCHOLOGICAL TESTING?

Unique Contribution To The Diagnostic PuzzleUnique Contribution To The Diagnostic Puzzle

UNIQUE CONTRIBUTION OF UNIQUE CONTRIBUTION OF NEUROPSYCHOLOGICAL TESTINGNEUROPSYCHOLOGICAL TESTINGTO CONCUSSION MANAGEMENTTO CONCUSSION MANAGEMENT

N=215, MANOVA p<.000000

Testing revealscognitive deficitsin asymptomaticathletes within 4 days post-injury

UNIQUE CONTRIBUTION OF UNIQUE CONTRIBUTION OF NEUROPSYCHOLOGICAL TESTINGNEUROPSYCHOLOGICAL TESTINGTO CONCUSSION MANAGEMENTTO CONCUSSION MANAGEMENT

MANOVA p<.000000 (ImPACT Test Battery)(ImPACT Test Battery)

Reaction Time Processing Speed

VALUE ADDED OF VALUE ADDED OF NEUROPSYCHOLOGICAL TESTINGNEUROPSYCHOLOGICAL TESTING

201 concussed High School and collegiate athletes tested with 2 days of injury. Abnormal performance determined by RCI’s (van Kampen, 2004).

NP testing increasesDiagnostic yield to88%

Either

Neuropsych

Symptoms %declinedFrom baseline

Cannot gauge severity of injury based on initial Cannot gauge severity of injury based on initial

symptomssymptoms

Acute recovery may not be a linear processAcute recovery may not be a linear process

WHY USE NEUROPSYCHOLOGICAL TESTING?WHY USE NEUROPSYCHOLOGICAL TESTING?

Mild or Mild or ““DingDing”” Injuries May be Significant Injuries May be Significant

64 high school athletes with Grade 1 concussion64 high school athletes with Grade 1 concussion

24 Non-injured control subjects24 Non-injured control subjects

All athletes diagnosed with All athletes diagnosed with ””dingding””Confusion, amnesia, signs/symptoms cleared Confusion, amnesia, signs/symptoms cleared

within 15 minuteswithin 15 minutesNo athlete in sample sustained LOCNo athlete in sample sustained LOC

No athlete returned to contestNo athlete returned to contest

ImPACT evaluation obtained at baseline, 36 hours, ImPACT evaluation obtained at baseline, 36 hours,

4 days and 7 days post-concussion4 days and 7 days post-concussion

Concussion and Memory DysfunctionConcussion and Memory Dysfunction

Lovell et al., J. Neurosurgery, 2003.

Memory Impairment Following Memory Impairment Following ““Mild ConcussionMild Concussion””

Time of Evaluation

Lovell, Collins, Iverson et al., J. Neurosurgery, 98, 2003

And Now for Something Completely Different…

Effort: (Green, et al., Brain Injury, 2001)• >50% of variance in scores was due to

poor effort

• Education = 11%

• Age = 4%

Symptom Validity Testing (Bush, et al NAN, ACN),:

“it is necessary to evaluate symptom validity objectively in any neuropsychological assessment”

And Now for Something Completely Different…

• Daubert v. Merrell Dow Pharmaceuticals (1993) held that trial judges should permit expert scientific testimony only when “the reasoning or methodology underlying the testimony is scientifically valid, and …properly can be applied to the facts in issue.”

And Now for Something Completely Different…

Effort Tests:

• Rey 15-item, Dot Counting

• Test of Memory Malingering (TOMM)

• Word Memory Test (WMT)

• CARB (Comp. Assess. Response Bias)

• Validity Indicator Profile (VIP)

• MMPI-2

Assessment of Effort - Rationale (Schatz & Sucharski, in some stage of submission)

• Little data on effort at time of assessment:– “…speculation that athletes are often motivated to underreport

symptoms so they can return to competition” (Echemendia & Cantu, 2003)

– “…motivated to underreport subjective symptoms to hasten the return to competition.” (Erlanger, et al., 2003)

– “An athlete’s apparent fear of removal from a game or of losing his or her position on the team may tempt some athletes to deny or underreport postconcussive symptoms.” (Lovell, 1998)

– “Ostensibly, symptom minimization occurs in hopes of a faster return to the playing field, court or ice.” (Lovell, et al., 2002)

– “it is more likely that suspicion regarding the use of the test results and/or general disinterest and apathy could impact the accurate measurement of cognitive ability at baseline” (Echemendia & Cantu, 2003)

Assessment of Effort - Rationale

• Bailey, Echemendia, Arnett (2006), JINS– Assigned to High Motivation at Baseline (HMB) and Suspect

Motivation at Baseline (SMB) groups based on whether baseline performance fell one or more standard deviations from the mean of the given measure

– N=26 to 33, Pre, Post-Mild TBI, Trails, Dig Span, Stroop

– In repeated measures ANCOVA (that removed achievement performance), the SMB groups demonstrated greater improvement than the HMB groups for the Trail Making Test A & B ( TMT-A & B), Digit Span, and Stroop-Color Word (Stroop-CW) tests.

Assessment of Effort - Methodology

• 155 Participants: 77 athletes, 78 from human subjects pool

• Administered Word Memory Test (WMT) and ImPACT

• Only looked at Baseline performance

Assessment of Effort - Methodology

Word Memory Test (WMT):

1. Show word pairs (e.g., Boy-Girl)

2. Show “cue” word (e.g., Boy)

3. Show “target” choices (e.g., Girl, Truck)

TBI Patients 95%+

Assessment of Effort - Incidence

• Incidence of Poor Effort

Assessment of Effort - Effects

What about “Sandbagging”?

"Players are smart. They know that if they have a concussion and score badly that, 'I'm going to be taken out. It's going to affect my livelihood,' " Amen said. "I've had a number of players tell me they purposely do bad on the testing to start so if they get a concussion it doesn't affect them. 

What about “Sandbagging”?

"They have these new (brain) tests we have to take. Before the season, you have to look at 20 pictures and turn the paper over and then try to draw those 20 pictures. And they do it with words, too. Twenty words, you flip it over, and try to write those 20 words."Then, after a concussion, you take the same test and if you do worse than you did on the first test, you can't play. So I just try to do badly on the first test.”-Peyton Manning

What about “Sandbagging”?

"They have these new (brain) tests we have to take. Before the season, you have to look at 20 pictures and turn the paper over and then try to draw those 20 pictures. And they do it with words, too. Twenty words, you flip it over, and try to write those 20 words."Then, after a concussion, you take the same test and if you do worse than you did on the first test, you can't play. So I just try to do badly on the first test.”-Peyton Manning

What about “Sandbagging”?

Erdal (2012):

75 athletes instructed to perform more poorly on the ImPACT than they had at baseline without reaching threshold on the validity indicators

8/75 (11%) of athletes were able to avoid detection of “sandbagging”

What about “Sandbagging”?

Schatz & Glatts (2012):

Coached versus naive malingering:1.Do Best2.Sandbag3.Sandbag, but...

What about “Sandbagging”?

Schatz & Glatts (2012):

What about “Sandbagging”?

Schatz & Glatts (2012):

Validation of MilliSecond Accuracy in MicroComputers (unpublished data)

PhotoCell

Internal Timer

External Timer

Mouse-click=stops both

Compare…Calculate Difference Score

Validation of MilliSecond Accuracy in MicroComputers (unpublished data)

Validation of MilliSecond Accuracy in MicroComputers

Validation of MilliSecond Accuracy in MicroComputers

Validation of MilliSecond Accuracy in MicroComputers

Validation of MilliSecond Accuracy in MicroComputers

Millisecond Accuracy?

Baseline to post-concussion Baseline to post-concussion computerscomputers

Caution when interpreting reaction Caution when interpreting reaction time and processing speedtime and processing speed

Use of software by cliniciansUse of software by clinicians