ScholarSpace at University of Hawaii at Manoa: Home ......Normal structural imaging Normal or...

ESTIMATING EMPLOYMENT STATUS IN A SAMPLE OF PARTICIPANTS WITH

TRAUMATIC BRAIN INJURY REFERRED FOR NEUROPSYCHOLOGICAL

ASSESSMENT FOR TREATMENT PLANNING OR FOR LITIGATION PURPOSES

A DISSERTATION SUBMITTED TO THE GRADUATE DIVISION OF THE

UNIVERSITY OF HAWAIʻI AT MĀNOA IN PARTIAL FULFILLMENT OF THE

REQUIREMENTS FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

IN

PSYCHOLOGY

DECEMBER 2014

By

James D. Larsen

Dissertation Committee:

Stephen Haynes, Chairperson

Elaine Heiby

Brad Nakamura

John Meyers

Joe Mobley

EMPLOYMENT STATUS FOLLOWING TBI 2

Abstract

Previous research has identified demographic and neuropsychological variables significantly

related to the amount of time that individuals take before returning to work following traumatic

brain injury (TBI). However, existing models do not identify variables significantly associated

with an individual’s current employment status as a function of time since TBI. The Meyers

Neuropsychological Battery (MNB) is a short battery of neuropsychological tests that assesses

the neuropsychological domains most commonly related to the likelihood that an individual will

be employed following a TBI. The goal of this study was to examine the degree to which scores

from the MNB, in combination with demographic information, predicted an individual’s

employment status as a function of time since TBI. Using archival data from a private practice

neuropsychology database of 192 male and female adults, exploratory and confirmatory

hierarchical regression modeling was used to examine the degree to which neuropsychological

test scores independently and incrementally accounted for variance in an individual’s

employment status, while considering time since injury and demographic variables. Regression

models were created using forward stepwise binary logistic regression on a sample of 96

participants and confirmed on three separate samples of participants taken from the same

database, including samples of litigants and non-litigants. Results showed that regression models

were able to correctly classify the employment status of between 78.6% and 88.5% of study

participants. These correct classification rates are higher than those attained by prediction models

examined in previously published research. The variables that were most consistently identified

as significant predictors of employment status were years of education, independent driving

status, premorbid occupation, Wechsler Adult Intelligence Scale-III Performance IQ score, and

the Overall Test Battery Mean. R2 values ranged from 0.28 to 0.40. Results show that post-TBI


employment status in the study sample could be predicted using a combination of scores from

the MNB and demographic information. These findings may be clinically useful when

determining the readiness to return to work of individuals who are recovering from TBI.


Table of Contents

List of Tables .................................................................................................................................. 9

List of Abbreviations .................................................................................................................... 10

Introduction ................................................................................................................................... 11

Psychological and Neuropsychological Consequences of TBI ................................................. 12

Difficulty Returning to Work Following TBI ........................................................................... 13

Benefits of Predicting Employment Status Following TBI ...................................................... 14

Demographic Variables Significantly Correlated with Employment Status Following TBI .... 15

Age at time of injury.............................................................................................................. 17

Premorbid occupation ............................................................................................................ 18

Years of education ................................................................................................................. 18

Number of symptoms ............................................................................................................ 19

Ethnic status........................................................................................................................... 20

Activities of daily living ........................................................................................................ 21

Neuropsychological Domains Significantly Correlated with Employment Status Following

TBI ............................................................................................................................................ 21

Memory ................................................................................................................................. 23

Attention ................................................................................................................................ 24

Visuospatial skills .................................................................................................................. 25

Executive functions ............................................................................................................... 25

General cognitive functioning ............................................................................................... 26

Language fluency .................................................................................................................. 27

Motor performance ................................................................................................................ 28

The Meyers Neuropsychological Battery .................................................................................. 29

Suitability of the MNB for Predicting Employment Status ...................................................... 31

Predicting Current Employment Status ..................................................................................... 33


Goals ............................................................................................................................................. 35

Method .......................................................................................................................................... 37

Participants ................................................................................................................................ 37

Group 1 .................................................................................................................................. 39

Group 2 .................................................................................................................................. 40

Group 3 .................................................................................................................................. 41

Group 4 .................................................................................................................................. 42

Procedure ................................................................................................................................... 45

Neuropsychological examination .......................................................................................... 45

Assessment domains, instruments and measures .................................................................. 46

Memory .............................................................................................................................. 46

RAVALT ........................................................................................................................ 46

Rey Complex Figure Test .............................................................................................. 48

Attention ............................................................................................................................ 50

Digit-Symbol Coding ..................................................................................................... 50

Trail Making Test, Part B ............................................................................................... 51

Visuospatial skills .............................................................................................................. 52

Judgment of Line Orientation ........................................................................................ 52

Block Design .................................................................................................................. 53

Executive functions ............................................................................................................ 54

Category Test ................................................................................................................. 54

Similarities ..................................................................................................................... 55

General cognitive function ................................................................................................. 56

WAIS-III index scores ................................................................................................... 56

Overall test battery mean ................................................................................................ 56


Language fluency ............................................................................................................... 56

COWAT ......................................................................................................................... 56

Motor performance ............................................................................................................ 57

Finger Tapping Test ....................................................................................................... 57

Data Reduction .......................................................................................................................... 58

Demographic Variables ......................................................................................................... 58

Employment status ............................................................................................................. 59

Ethnicity ............................................................................................................................. 59

Premorbid Occupation ....................................................................................................... 60

Independent driving status ................................................................................................. 60

Neuropsychological assessment measures ............................................................................ 61

Data Analysis ............................................................................................................................ 62

Goal 1 .................................................................................................................................... 64

Goal 2 .................................................................................................................................... 65

Goal 3 .................................................................................................................................... 67

Goal 4 .................................................................................................................................... 69

Goal 5 .................................................................................................................................... 70

Goal 6 .................................................................................................................................... 71

Goal 7 .................................................................................................................................... 72

Goal 8 .................................................................................................................................... 73

Results ........................................................................................................................................... 74

Basic Data ................................................................................................................................. 74

Initial Analyses .......................................................................................................................... 76

Study Goals ............................................................................................................................... 77

Goal 1 .................................................................................................................................... 77


Goal 2 .................................................................................................................................... 81

Goal 3 .................................................................................................................................... 81

Goal 4 .................................................................................................................................... 84

Goal 5 .................................................................................................................................... 89

Goal 6 .................................................................................................................................... 91

Goal 7 .................................................................................................................................... 94

Model 4 .............................................................................................................................. 94

Model 5 .............................................................................................................................. 97

Goal 8 .................................................................................................................................. 100

Model 6 ............................................................................................................................ 100

Model 7 ............................................................................................................................ 103

Discussion ................................................................................................................................... 109

A Parsimonious Model to Predict Employment Status ........................................................... 109

Sensitivity, Specificity, and Predictive Efficacy of a Parsimonious Model ............................ 111

The Addition of Demographic Predictors ............................................................................... 113

The Additive Value of Neuropsychological Variables ........................................................... 115

An Investigation of Model Performance ................................................................................. 118

Proportion of variance accounted for .................................................................................. 118

Percentage of cases correctly classified .............................................................................. 119

Sensitivity ............................................................................................................................ 120

Specificity ............................................................................................................................ 120

Positive Predictive Value .................................................................................................... 120

Negative Predictive Value ................................................................................................... 121

The Relationship Between Litigation Status and Model Performance ................................... 122

Model Creation in a Sample of Litigants ................................................................................ 122


Model Creation Using the Entire Study Sample ..................................................................... 124

The Difficulty of Predicting Employment Status .................................................................... 127

Limitations of the Current Study ............................................................................................. 129

Directions for Future Research................................................................................................ 134

References ................................................................................................................................... 138


List of Tables

1 Department of Defense TBI Severity Classification System .............................................11

2 Demographic Variables Significantly Correlated with Employment After TBI ...............16

3 Neuropsychological Domains Significantly Correlated with Employment After TBI ......22

4 Meyers Neuropsychological Battery Tests Organized by Neuropsychological Domain ..32

5 Demographic Variables of Interest in Groups 1-4 and the Full Study Sample .................44

6 Exploratory Stepwise Binary Logistic Regression Models Created ..................................63

7 Neuropsychological Variables of Interest in Groups 1-4 and the Full Study Sample .......75

8 Summary of the Two Steps Completed During Creation of Model 1 ...............................78

9 Summary of the Five Steps Completed During Creation of Model 1b..............................80

10 Summary of the Seven Steps Completed During Creation of Model 2 .............................83

11 Summary of the Seven Steps Completed During Creation of Model 3 .............................87

12 Statistical Performance of Models 1b, 2, and 3 in Groups 1 and 2....................................90

13 Statistical Performance of Models 1b through 3 in Groups 3 and 4 ..................................93

14 Summary of the Six Steps Completed During Creation of Model 4 .................................96

15 Summary of the Five Steps Completed During Creation of Model 5................................99

16 Summary of the Five Steps Completed During Creation of Model 6..............................102

17 Summary of the Eight Steps Completed During Creation of Model 7 ............................105

18 Variables Retained During Regression Analyses in Models 1 through 7 ........................107

19 Statistical Performance of Models 1b through 7 ..............................................................108


List of Abbreviations

Abbreviation Definition

COWAT Controlled Oral Word Association Test

FSIQ Full-scale Intelligence Quotient

MNB Meyers Neuropsychological Battery

OTBM Overall Test Battery Mean

PIQ Performance Intelligence Quotient

RAVLT Rey Auditory Verbal Learning Test

SD Standard deviation

TBI Traumatic Brain Injury

VIQ Verbal Intelligence Quotient

WAIS-III Wechsler Adult Intelligence Scale, 3rd Edition


Estimating Employment Status in a Sample of Participants with Traumatic Brain Injury Referred

for Neuropsychological Assessment for Treatment Planning or for Litigation Purposes

According to the American Congress of Rehabilitation Medicine (Kay et al., 1993),

Traumatic Brain Injury (TBI) has been defined as any traumatically induced physiological

disruption of brain function as evidenced by any period of loss of consciousness, any loss of

memory for events immediately before or after the accident, any alteration in mental state at the

time of the event, or any focal neurological deficit. While multiple classification systems exist to

rate the severity of TBI, most are comparable. In 2009, the Department of Veterans Affairs and

Department of Defense published Table 1 for use as a tool to rate the severity of a TBI (The

Management of Concussion/TBI Working Group, 2009). In using this classification system, each

injury is assigned the most severe classification for which at least one of the listed criteria is met.

Table 1

Department of Defense TBI Severity Classification System.

Mild TBI Moderate TBI Severe TBI

Normal structural imaging Normal or abnormal structural imaging. Severity based

on other criteria.

LOC = 0-30 minutes LOC > 30 minutes and <

24 hours LOC > 24 hours

AOC = a moment up to 24

hours AOC > 24 hours. Severity based on other criteria.

PTA = 0 to 1 days PTA > 1 and < 7 days PTA > 7 days

GCS = 13-15 GCS = 9-12 GCS < 9

Note: GCS scores represent the best score taken during the first 24 hours after injury.

TBI = Traumatic Brain Injury; LOC = loss of consciousness; AOC = alteration of

consciousness; PTA = posttraumatic amnesia; GCS = Glascow Coma Scale.


TBI is a growing concern in the United States. In 2009, approximately 3.5 million

individuals in the United States received inpatient or outpatient medical treatment for a primary

or secondary diagnosis of TBI. In the same year, an additional 52,695 individuals in the United

States died due to causes directly related to the effects of TBI (Coronado et al., 2012). Mild TBI

is thought to account for roughly 75% of all head injuries in the United States (Faul, Wald, Xu &

Coronado, 2010).

Psychological and Neuropsychological Consequences of TBI

The psychological and neuropsychological impact of TBI can be severe. Several notable

meta-analyses (Belanger, Curtiss, Demery, Lebowitz, and Vanderploeg, 2005; Frencham, Fox,

and Maybery, 2005; Rohling, Binder, Demakis, Larrabee, Ploetz, and Langhinrichsen-Rohling,

2011) have demonstrated that mild TBI can affect a spectrum of neuropsychological domains

during the first three months following the injury, including processing speed, working memory,

attention, memory, and executive functions. Additional research (Bigler, Farrer, Pertab, James,

Petrie, and Hedges, 2013; Iverson, 2010; Pertab, James, & Bigler, 2009) has shown that the

cognitive deficits described by Frencham et al. persist beyond three months in as many as 24%

of victims of mild TBI, and may persist for as long as ten years (Ponsford, Downing, Olver,

Ponsford, Acher, Carty, & Spitz, 2014). In a sample of individuals treated at a rehabilitation

hospital, Whelan-Goodinson, Ponsford, Johnston, and Grant (2009) showed a significant

increase in rates of depressive disorders, generalized anxiety disorder, posttraumatic stress

disorder, panic disorder, and phobias following mild to severe TBI when compared to premorbid

rates of those disorders in the same sample. De Guise, LeBlanc, Tinawi, Lamoureux, and Feyz

(2012) found that up to two weeks after injury, victims of mild TBI had significantly elevated

scores on both the Beck Depression Inventory and the Beck Anxiety Inventory when compared


to a normative sample. Konrad et al. (2011) showed that when compared to healthy controls,

victims of mild TBI experienced a significantly higher number of depressive symptoms as

measured by the Beck Depression Inventory and a significantly higher level of impairment in

daily life as measured by a 25-item questionnaire derived from the Rivermead Post Concussion

Symptoms Questionnaire (King, Crawford, Wenden, Moss, & Wade, 1995). The identified

between-group differences remained significant up to 6 years post-injury. Approximately 50% of

victims of TBI experience high levels of apathy, which has been related to poor rehabilitation

outcome, loss of social autonomy, financial problems, vocational loss, cognitive decline, and

caregiver distress (Arnould, Rochat, Azouvi, & Van der Linden, 2013). In a sample of 205

individuals with moderate to severe TBI, approximately 65% of individuals were found to

experience sleep disturbance in the month following injury (Nakase-Richardson et al., 2013).

These psychological and neuropsychological consequences can combine to result in an inability

to continue previously established routines, abilities, and roles, including employment

(Grauwmeijer, Heijenbrok-Kal, Haitsma, & Ribbers, 2012; Robertson, 2008).

Difficulty Returning to Work Following TBI

Difficulty returning to work following TBI is a widespread problem. Parks, Diaz-

Arrastia, Gentilello, and Shafi (2010) followed a group of 572 individuals in the State of

Colorado who had suffered a TBI between 1996 and 1999. Of the 381 members of the cohort

who were employed at the time of injury, only 69% were employed one year post-injury and

only 74% were employed three years post-injury. Similarly, the Traumatic Brain Injury Model

Systems estimated that 63% of individuals are employed at the time they experience a TBI,

whereas employment rates drop to 28% after one year (The Traumatic Brain Injury Model

Systems, 2010). Grauwmeijer, Heijenbrok-Kal, Haitsma, and Ribbers (2012) found that in a


cohort of 113 patients hospitalized following a moderate to severe TBI, employment rates

climbed steadily to 55% during the first year post-injury, but showed little improvement during

the following two years. Other studies have shown that those who return to work following TBI

often have difficulty maintaining their employment (Fraser, Machamer, Temkin, Dikmen, &

Doctor, 2006; Yasuda, Wehman, Targett, Cifu, & West, 2001), especially when medical

symptoms and emotional dysregulation persist (Artman & McMahon, 2013). Sigurdardottir,

Andelic, Roe, and Schanke (2013) showed that lack of employment during the first 5 years

following TBI is significantly related to an increase in depressive symptoms. In such a case,

depressive symptoms and unemployment would have a reciprocal relationship in which each

increases the likelihood of the other.

There are also collateral effects of not returning to work following a brain injury.

Wehman, Targett, West, and Kregel (2005) remind us that working is tied to other activities that

promote recovery, such as a sense of purpose, a reason to leave the house, and the creation or

maintenance of friendships. Cattelani, Tanzi, Lombardi, and Mazzucchi (2002) point out that a

failure to return to pre-injury employment or work status can result in decreased quality of life

for victims of TBI and their families.

Benefits of Predicting Employment Status Following TBI

Sherer, Novack, Sander, Struchen, Alderson, and Thompson (2002) recognized that one

of the purposes of neuropsychological assessment following TBI is the prediction of the degree

and latency of return to normal cognitive functioning. They proposed that the ability to return to

work was an ecologically valid measure of general neuropsychological functioning. Robertson

(2008) described another goal of neuropsychological assessment when he stated that a prominent

concern for many victims of TBI is how soon they can expect to resume previous roles, such as


employment. The ability to use neuropsychological assessment scores to help predict

employment status following a TBI would help to accomplish both of the above-mentioned core

purposes of neuropsychological assessment. However, Robertson points out another, perhaps

more important reason for understanding return to work following TBI: A better understanding

of the factors that contribute to a return to employment may help identify interventions that

enhance the recovery process.

Demographic Variables Significantly Correlated with Employment Status Following TBI

Current models for predicting employment status following TBI are moderately effective,

with correct classification rates between 65% and 77% (Drake, Gray, Yoder, Pramuka, &

Llewellyn, 2000; Fleming, Tooth, Hassell, & Burchan, 1999; Guerin, Kennepohl, Leveille,

Dominique, & McKerral, 2006; Kreutzer et al., 2003; MacMillan, Hart, Martelli, & Zasler, 2002;

Simpson & Schmitter-Edgecombe, 2002), and are improving our understanding of the factors

moderating return to work (Ownsworth & McKenna, 2004; Shames, Treger, Ring, & Giaquinto,

2007). Recent research has identified a number of demographic variables that are significantly

correlated with employment status following TBI, as outlined in Table 2.


Table 2

Demographic Variables Significantly Correlated with Employment After TBI

Study

Demographic Variables Identified

Age at

Injury

Premorbid

Occupation

Level of

Education

Number

of

Symptoms

Ethnic

Minority

Status

Activities

of Daily

Living

Arango-Lasprilla (2009) (+) (-)

Arango-Lasprilla (2011) (-)

Chamelian (2004) (-)

Drake (2000) (+) (+) (-)

Flemming (1999) (+) (-) (+)

Gary (2009) (+) (+) (+) (-)

Grauwmeijer (2012) (-) (-)

Guerin (2006) (-) (-)

Hanlon (1999) (-)

Johansson (2001) (+)

Ketchum (2012) (+) (+)

Keyser-Marcus (2002) (-) (+)

Kreutzer (2003) (-) (+) (-)

Machamer (2005) (+)

Ownsworth (2004) (-) (+) (+) (-)

Schonberger (2011) (-) (+) (+)

Shames (2002) (-) (+) (+)

Simpson (2002) (+)

van der Horn (2013) (-)

Walker (2006) (+)

Note: Boxes marked with a (+) signify that the corresponding study found a statistically significant

positive relationship between the corresponding demographic variable and employment status

following TBI. Boxes marked with a (-) signify that the corresponding study found a statistically

significant negative relationship between the corresponding demographic variable and employment

status following TBI. Studies are identified by the name of the first author and year of publication

in order to conserve space. See References for complete article reference information.


Age at time of injury. A number of studies have found a significant, inverse relationship

between a person’s age at the time they acquire a TBI and their ability to return to work

afterward, with older individuals taking longer to return to work and having more difficulty

maintaining stable employment following a TBI than younger individuals (Grauwmeijer,

Heijenbrok-Kal, Haitsma, and Ribbers, 2012; Guerin et al., 2006; Hanlon, Demery, Martinovich,

& Kelly, 1999; Jourdan et al., 2013; Keyser-Marcus et al., 2002; Ketchum et al., 2012; Kreutzer,

et al., 2003). Schonberger, Ponsford, Olver, Ponsford, and Wirtz (2011) suggested that the

significant correlation between age and a failure to return to work may be due to the fact that

older victims of TBI chose to retire rather than return to work following their injury. However, in

certain cases a significant positive correlation has been found between a person’s age at the time

they are injured and their ability to return to work afterwards. Drake et al. (2000) found that

older active-duty members of the armed forces were able to return to full duty more quickly

following a TBI. Similarly, Machamer, Temkin, Fraser, Doctor, and Dikmen (2005) found that

older individuals maintained more steady employment following a TBI, an effect the authors

ascribed to the benefits of being well-established in a career at the time of injury.

Guerin et al. (2006) collected demographic, neurological, psychological, and

environmental data from 110 individuals receiving treatment for mild TBI at a major

rehabilitation center in Montreal, Canada. Logistic regression showed that age at time of injury

was the variable most strongly related to whether or not participants were able to return to at

least part-time employment at the end of the treatment program. In another study (Hanlon et al.,

1999), the participant's age at the time of injury was identified as the only demographic variable

significantly correlated with post-injury employment status. Research has also demonstrated a


significant correlation between age at the time of injury and the stability of employment

following TBI (Kreutzer et al., 2003; Machamer et al., 2005).

Premorbid occupation. The type of employment held by an individual at the time of

their injury has also been investigated as a possible predictor of whether or not he or she will

return to work afterwards (Shames et al., 2007). Walker, Marwitz, Kreutzer, Hart, and Novack

(2006) studied a Traumatic Brain Injury Model Systems database cohort of 1341 individuals who

had been hospitalized for TBI. They divided participants into three groups based on their pre-

injury employment, categorizing each individual as a professional/managerial worker, a skilled

worker, or a manual laborer. They found that between 10 and 14 months post-injury, skilled

workers were just over 1.5 times as likely to have returned to work as manual laborers, while

professional and managerial workers were nearly 3 times as likely to have returned to work as

manual laborers. Two additional studies of return to work rates following TBI in minority

populations also found that pre-injury occupation significantly predicted post-injury occupation

regardless of minority status (Arango-Lasprilla et al., 2009; Gary et al., 2009). Andelic, Stevens,

Sigurdardottir, Arrango-Lasprilla, and Roe (2012) found that individuals who were unemployed

prior to experiencing a TBI were 95% less likely to find employment following their injury than

those who had previously been employed. Schonberger, Ponsford, Olver, Ponsford, and Wirtz

(2011) also found that premorbid employment (i.e. individuals who were employed vs. those

who were unemployed) was a significant (p < 0.01) predictor of employment status one year

after a TBI.

Years of education. Simpson and Schmitter-Edgecombe (2002) used a discriminant

function analysis to identify which combination of neuropsychological measures and

demographic variables were mostly strongly correlated with employment status following a TBI.


The 61 participants in their study were injured an average of ten and a half years before data

collection. Results showed that level of education was the demographic variable most strongly

correlated with employment status at the time of data collection, with unemployed individuals

having significantly (p < 0.05) fewer years of education than those who were employed. Kruetzer

and colleagues (2003) made the observation that those who have graduated from high school

were nearly twice as likely to find employment during the four years following a TBI than those

who did not graduate from high school. Similarly, Keyser-Marcus and colleagues (2002) found

that individuals with higher levels of education were significantly more likely to return to work

within one year following a TBI (p < .01; odds ratio = 1.38). Using structural equation modeling,

Schonberger, Ponsford, Olver, Ponsford, and Wirtz (2011) found that the number of years of

formal education completed before sustaining a TBI was a significant (p < 0.01) predictor of

employment status one year after injury. Similarly, Ketchum et al. (2012) used a logistic

regression model to show that pre-injury employment status accounted for a significant (p <

0.01) amount of variance in employment status 1 year after TBI in a population of 418 Hispanic

individuals hospitalized for TBI between 1990 and 2009.

Number of symptoms. Fleming et al. (1999) used the hospital records of 209 TBI

patients admitted between 1991 and 1995 to identify which variables would be most predictive

of employment status two to five years post-injury. Among the measures collected were scores

from the Disability Rating Scale (Rappaport, Hall, Hopkins, Belleza, & Cope, 1982), an

observational rating scale that quantifies functional disability across categories following TBI.

Discriminant function analysis showed that the Disability Rating Scale total score was a

significant predictor of employment status following TBI (F1,200 = 12.80; p < .01), with lower

scores predicting a greater chance of being employed. In a sample of 242 adults with mild to


severe TBI, van der Horn, Spikman, Jacobs, and van der Naalt (2013) found that individuals who

did not return to competitive employment after mild TBI also experienced a significantly (p <

0.01) higher number of self-reported depression- and anxiety-related symptoms. Similar

relationships between the number of subjective symptoms immediately following a TBI and

post-injury employment status have been reported elsewhere (Chamelian & Feinstein, 2004;

Drake et al., 2000; Guerin et al., 2006).

Ethnic status. In a review of studies investigating variables and measures that might

have been significantly correlated with employment status following TBI, Ownsworth and

McKenna (2004) found that studies with robust methodologies all found that ethnic minority

groups were at a significant disadvantage when seeking employment following TBI. Numerous

other empirical studies have found significant correlations between minority status and

unemployment following TBI (Gary et al., 2009; Kreutzer et al., 2003). Notably, Arango-

Lasprilla and colleagues (2009) followed a Traumatic Brain Injury Model Systems cohort of 633

individuals, 219 of whom were ethnic minorities, for three years following TBI. Employment

status was assessed each year during the three-year follow-up period. They found that when

compared to White Americans, members of ethnic minority groups were between two and three

and a half times more likely to be unemployed or unstably employed during the three years

following a TBI. These differences were true even when controlling for other variables such as

premorbid employment status, age, marital status, level of education, cause of injury, loss of

consciousness, and general level of impairment. A follow up study (Arango-Lasprilla, Ketchum,

Lewis, Krch, Gary, & Dodd, 2011) found that Whites continued to find competitive employment

at higher rates than Hispanics and Blacks five years after a moderate to severe TBI. The


researchers concluded that it is critical to consider ethnic minority status when predicting

employment status following TBI.

Activities of daily living. In their study of community integration and employment

outcomes following participation in a TBI rehabilitation program in Australia, Fleming and

colleagues (1999) assessed each participant using the Modified Barthel Index (Shah, Vanclay, &

Cooper, 1989), an instrument which rates level of functioning in 10 areas of activities of daily

living including personal hygiene, bathing, feeding, use of the toilet, stair climbing, dressing,

bowel control, bladder control, ambulation, and transferring from a chair to a bed. They found

that individuals with higher scores on the Modified Barthel Index were significantly more likely

to be employed between two and five years post-injury than individuals with low scores, and that

scores on the Index significantly (p < 0.01) predicted post-injury employment status. In another

study, Johansson and Bernspang (2001) showed that occupational therapy assessments were

useful in predicting work status following a TBI, mostly through their collection of in-depth

information regarding activities of daily living. Recently, Forslund, Roe, Arango-Lasprilla,

Sigurdardottir, & Andelic (2013) showed that 2 years after a moderate to severe TBI, individuals

who were driving independently were significantly more likely to be employed than those who

were not driving independently (p < 0.01, odds ratio = 8.4).

Neuropsychological Domains Significantly Correlated with Employment Status Following

TBI

In addition to the demographic variables discussed above, a number of

neuropsychological measures have been investigated in order to examine their validity and

clinical utility in predicting employment status after a TBI. While there has been little

consistency in the assessment instruments used across studies, certain general domains of


neuropsychological functioning have often been the focus of these investigations, as outlined in

Table 3. Although the exact definitions of these domains vary across studies, the differences are

not extensive enough to prevent comparisons. Some of the most commonly investigated domains

are discussed below.

Table 3

Neuropsychological Domains Significantly Correlated with Employment After TBI

Study

Neuropsychological Domains Identified

Memory Attention Visuo-

spatial

Executive

Functions

General

Cognitive

Function

Language

Fluency Motor

Cifu (1997) (+)

Drake (2000) (+) (+) (+)

Fraser (2006) (+) (+)

Han (2009) (+) (-)

Hanlon (1999) (+) (+) (+)

Johansson (2001) (+)

Machamer (2005) (+) (+) (+) (+)

Ownsworth (2004) (+) (+) (+)

Ryu (2010) (+) (+)

Sherer (2002) (+) (+) (+)

Sigurdardottir (2009) (+)

Spitz (2012) (+) (+) (+)

Zakzanis (2013) (+)

Note: Boxes marked with a (+) signify that the corresponding study found a statistically

significant positive relationship between the corresponding neuropsychological domain

and employment status following TBI. Boxes marked with a (-) signify that the

corresponding study found a statistically significant negative relationship between the

corresponding neuropsychological domain and employment status following TBI. Studies

are identified by the name of the first author and year of publication only in order to

conserve space. See References for complete article reference information.


Memory. Memory impairment is one of the most frequently reported and researched

effects of TBI (Vakil, 2005) and has been frequently studied as a predictor of post-injury

employment status. Scores from a wide variety of commonly used memory tests have been

identified as significant predictors of employment status following TBI. Drake et al. (2000)

found significant (p < 0.05) between group differences in the California Verbal Learning Test-II

(Delis, Kramer, Kaplan, & Ober, 2000) long-delay free recall score when comparing active duty

military members who were or were not able to return to full duty following a TBI, with those

who returned to full duty attaining higher scores. Using a statistical procedure known as optimal

data analysis, Han et al. (2009) found that in a sample of 52 active duty military personnel, the

percentage of change between the California Verbal Learning Test-II short-delay free recall and

long-delay recall scores, as well as the total recognition hits score, accounted for a significant

amount of variance (p = 0.01) in employment status following TBI. Other measures of memory

that have been found useful in predicting employment status following TBI include the Selective

Reminding Test (Buschke, 1973; Machamer et al., 2005; Fraser et al., 2006) the Rivermead

Behavioural Memory Test (Johansson & Bernspang, 2001; Wilson, Cockburn, Baddeley, &

Hiorns, 1988), and certain subtests of the Wechsler Memory Scale – Revised (Cifu et al., 1997;

Hanlon et al, 1999; Wechsler, 1987). While not directly linked to employment status, Spitz,

Ponsford, Rudzki, and Maller (2012) found that the total score and the 20-minute delayed recall

score from the Rey Auditory Verbal Learning Test (RAVLT) and the Immediate Memory Index

score from the Wechsler Adult Intelligence Scale – III (WAIS-III; Wechsler, 1997) were

significantly different between healthy controls and individuals who had experienced a moderate

to severe TBI. These differences were significant (p < 0.01) at 3, 6, and 12 months post-injury,

with the exception of the WAIS-III Immediate Memory Index score at 12 months.


Attention. Attention refers to the process by which a person orients to, selects, and

maintains focus on information to make it available for cortical processing (Zillmer, Spiers, &

Culbertson, 2008). It is another neuropsychological domain frequently investigated in studies of

employment status following TBI. In the study mentioned above, Hanlon et al. (1999) found that

scores on the Trail Making Test part B (Reitan, 1958), a commonly used measure of switching

attention between two tasks, were significantly correlated with employment status following TBI

(r = 0.30; p < 0.01). In addition, Machamer and colleagues (2005) found that scores on the Trail

Making Test part B collected one month post-injury were significantly different between groups

of TBI patients who had worked less than 50% of the time, between 50% and 89% of the time,

and 90% of the time or more since their injury.

Ryu, Cullen, and Bayley (2010) assessed 87 TBI patients participating in an inpatient

rehabilitation program. Their test battery included the Digit-Symbol Coding subtest from the

WAIS-III to measure attention and processing speed. Later statistical analysis showed that of all

the neuropsychological tests administered, scores from the Digit-Symbol Coding subtest were

most significantly correlated with employment status at one year post-injury (p = 0.03) and

showed moderate effect size (r = 0.63). This finding has been echoed in other studies, which

have also found performance on the Digit-Symbol Coding subtest to significantly predict

employment status following TBI (Fraser et al., 2006; Machamer et al., 2005).

Fatigue, as measured by the Fatigue Severity Scale (Krupp, LaRocca, Muir-Nash, &

Steinberg, 1989), which is related to deficits in attention, has also been linked to employment

status at one-year post-TBI, with high amounts of fatigue significantly related to poor

employment outcomes, especially when fatigue persists late in the recovery process

(Sigurdardottir, Andelic, Roe, & Schanke, 2009).


Visuospatial skills. Visuospatial ability has been described as the ability to process the

position, direction, or movement of objects or points in space (Elias & Saucier, 2006). Two of

the studies mentioned above present evidence that measures of visuospatial performance are

significantly correlated with employment status following a TBI. Hanlon et al. (1999) found that

scores from the Judgment of Line Orientation test (Benton, Varney, & Hamsher, 1978) were

moderately correlated with employment status at one year post-injury (r = 0.25), with higher

scores being associated with more favorable employment outcomes. Additionally, Ryu and

colleagues (2010) found that scores from the WAIS-III Block Design subtest were predictive of

employment status one year after injury. These findings are supported by Ownsworth and

McKenna (2004), whose literature review found moderate support for visuospatial skills as a

significant predictor of employment status following TBI across studies.

Executive functions. Executive functions include higher order regulatory and

supervisory functions, such as planning, mental flexibility, attentional allocation, and inhibitory

control (Zilmer et al., 2008). Empirical studies have shown that executive functions do not

represent a single, unitary construct (Parkin, 1998; Salthouse, 2005; Varney & Stewart, 2004),

and have disputed the claim that they reside primarily in the frontal lobes (Alvarez & Emory,

2006; Meyers & Rohling, 2009). Instead, executive function can be conceptualized as a

“macroconstruct” (Zelazo, Carter, Reznick, & Frye, 1997, p. 219) that describes the coordination

of multiple psychological processes to allow an organism to solve complex problems across a

variety of contexts. The term will be used in the current study to describe a single

neuropsychological domain in order to maintain consistency with previously published research.

In a review of scientific literature published between 1980 and 2003, Ownsworth and McKenna

(2004) found that employment status following TBI was found to share a significant correlation


with executive functions more consistently than with any other domain of neuropsychological

functioning addressed in their literature review.

In a study of active-duty military personnel, Drake and colleagues (2000) administered

the Map Planning Test (Ekstrom, French, Harman, & Derman, 1976) to 121 participants an

average of three months following a mild TBI. The Map Planning Test asks participants to find

the shortest path between two spots on a matrix-type grid while avoiding certain obstacles. They

found that the number of maps completed was significantly (p = 0.05) different between service

members who were able and those who were not able to return to full work duties six months

after injury. They also noted that those who were unable to return to full work duties were rated

as having significantly (p < 0.01) higher executive dysfunction by the Neurobehavioral Rating

Scale (Levin et al., 1987). Spitz, Ponsford, Rudzki, and Maller (2012) assessed executive

functioning performance in 111 individuals at 3, 6, and 12 months after a moderate-to-severe

TBI. Their battery included the Zoo Map test, the Trail Making Test Part B, the Controlled Oral

Word Association test, and the Working Memory Index from the WMS-III. They found that

gains in executive functioning performance as measured by these instruments mirrored gains in

functional outcomes as measured by the Mayo-Portland Adaptability Inventory during the first

year following injury.

General cognitive functioning. In their literature review, Ownsworth and McKenna

(2004) found moderate support for using “general intellectual or global cognitive functioning”

(p. 774) to predict employment status following TBI. For the purpose of this paper, all measures

that represent combined neuropsychological performance across domains will be said to

represent general cognitive function. Perhaps the most common measures of general cognitive

function used in the literature to predict employment status following TBI have been the various


WAIS-III index scores. Past research has found these scores to be significantly correlated with

post-TBI employment status (Han et al., 2009; Machamer et al., 2005).

In their study of a Traumatic Brain Injury Model Systems cohort of 388 adults with TBI,

Sherer, Sander, Nick, High, Malec, and Rosenthal (2002) combined scores from 16

neuropsychological tests to create a single score which they called “overall cognitive status” (p.

188). The tests comprising this score measured a variety of neuropsychological domains,

including motor performance, verbal fluency, visuospatial performance, memory, attention, and

executive function. Results showed that the overall cognitive status scores obtained one month

after TBI were significantly (p = 0.02) correlated with employment status at one year post-injury,

and remained so even when controlling for other significant predictors of functional outcome

such as severity of injury, education, and pre-injury productivity status.

Language fluency. In their study of employment status in active duty military personnel

with brain injuries, Drake and colleagues (2000) retained the total number of correct words on

the Controlled Oral Word Association Test (COWAT; Benton & Hamsher, 1989) as a measure

of language fluency in their predictive model. Zakzanis, McDonald, and Troyer (2013) recoded

COWAT scores from 28 patients who had incurred severe TBI and 54 healthy controls to

produce both semantic fluency and phonemic fluency scores. They found that both semantic

fluency and phonemic fluency scores were significantly different between victims of TBI and

healthy controls, with semantic fluency showing larger effect sizes (d = 1.53) than phonemic

fluency (d = 0.62). The COWAT was also one of the tests included in the overall cognitive status

score created by Sherer, Sander, Nick, High, Malec, and Rosenthal (2002), which was also

predictive of employment status one year after TBI. Spitz, Ponsford, Rudzki, and Maller (2012)


found that individuals who had undergone a moderate to severe TBI had significantly (p < 0.01)

lower COWAT scores than healthy controls at 3, 6, and 12 months post-injury.

Motor performance. Motor performance represents an individual’s ability to

demonstrate bilateral fine muscle control in the upper and lower extremities (Zillmer et al.,

2008). Machamer et al. (2005) investigated motor performance in individuals who had

experienced different levels of job stability following TBI. They found that in the first three to

five years after TBI, dominant-hand scores on the Finger Tapping Test (Reitan & Wolfson,

1993) were significantly (p < 0.01) different between those who had worked 50% of the time or

less, those who had worked 51-89% of the time, and those who had worked 90% of the time or

more, with individuals who achieved higher scores securing more consistent employment.

Sherer, Sander, Nick, High, Malec, and Rosenthal (2002) also included the Grooved Pegboard

Test (Klove, 1963), a test of fine motor control, in the calculation of their overall cognitive status

score.

While there is only moderate agreement across studies on the specific assessment

instruments used, the demographic and neuropsychological categories mentioned above

represent a consensus about which variables have been found to be most significantly associated

with employment status following TBI. Any standardized assessment battery wishing to

successfully predict employment status following TBI would need to measure most or all of

these domains in order to maximize its predictive efficacy. However, cost-efficiency is an

important concern in neuropsychological assessment and the use of a short battery to make valid

predictions could prove to be both efficacious and clinically useful.


The Meyers Neuropsychological Battery

The Meyers Neuropsychological Battery (MNB; Volbrecht, Meyers, & Kaster-

Bundgaard, 2000) is a short, flexible neuropsychological battery that takes approximately two

and a half hours to administer. The battery contains 16 core tests covering a variety of

neuropsychological domains. These tests include the Ward seven subtest short form of the

WAIS-III (Ward, 1990), the Animal Naming Test (Spreen & Strauss, 1998), COWAT (Benton &

Hamsher, 1989), One-minute Estimation (Benton, Van Allen, & Fogel, 1964), Dichotic

Listening (Roberts et al., 1994), Sentence Repetition (Spreen & Strauss, 1991), Judgment of Line

Orientation (Benton et al., 1978), Boston Naming Test (Kaplan, Goodglass, & Weintraub, 1983),

Finger Tapping (Reitan & Wolfson, 1993), Finger Localization (Benton, Hamsher, Varney, &

Spreen, 1983), Trail Making Test (Reitan, 1958), Token Test (De Renzi & Vignolo, 1962), the

Victoria Revision of the Category Test (Sherrill, 1987), Rey Complex Figure Test (Meyers &

Meyers, 1995), the RAVLT (Rey, 1964), and a forced-choice effort measure (for a description of

each measure, see the Methods section). The subtests included in the MNB were originally

selected based on the availability and range of normative data, demonstrated sensitivity to brain

injury, and the need for battery that measured multiple domains (Volbrecht et al., 2000). Many of

the subtests included in the MNB are available for public use, which reduces the cost of battery

administration. Lezak, Howieson, and Loring (2004) remind us that the time and cost of a

neuropsychological assessment can reduce access to neuropsychological services for many

patients. The short administration time and low cost of administering the MNB can be listed

among its significant contributions.

Meyers and Rohling (2004) demonstrated the validity and clinical utility of the MNB in

evaluating individuals who have experienced a mild TBI. They showed a correct classification


rate of 96.1% when discriminating between individuals with a mild TBI (n = 57) and those who

had been hospitalized for other reasons including chronic pain and depression (n = 103). They

also calculated an Overall Test Battery Mean (OTBM) by calculating the mean of all test T-

scores, which they found to have good test-retest reliability over a period of 12 to 14 months in a

TBI population (r = 0.86, Miller & Rohling, 2001).

Rohling, Meyers, and Millis (2003) demonstrated the discriminative validity of the

MNB’s OTBM score as a measure of general cognitive performance following TBI. A sample of

291 consecutive TBI referrals to a neuropsychological assessment practice was divided into six

groups based on injury severity as measured by length of loss of consciousness following the

injury. Analysis of variance showed that OTBM scores were significantly different between

severity groups (p < 0.01), and linear regression showed that injury severity accounted for 34%

of the variance in OTBM scores (r = -0.58). These results are similar to those reported for the

Halstead-Reitan Battery’s Impairment Index score, which showed a correlation of r = 0.59 with

the time needed to follow simple commands after a TBI (Dikmen, Machamer, Winn, & Temkin,

1995).

Meyers, Volbrecht, and Kaster-Bundgaard (1999) demonstrated the discriminative and

ecological validity of the MNB by using MNB scores to discriminate between those who were

currently driving (n = 230) and those who were unable to drive (n = 82) in a sample of 312

consecutive outpatient referrals to a neuropsychological assessment clinic. Participants

represented a wide range of common neuropsychological diagnostic groups. Discriminant

function analysis was able to distinguish significantly between the two groups (p < 0.01), and

resulted in a 94.4% correct classification rate.


The assessment of effort is also an essential part of any neuropsychological battery

measuring cognitive performance following TBI. The MNB contains nine identified measures of

patient effort during neuropsychological testing. These measures have shown 83% sensitivity in

a group of simulated malingerers, as well as 100% specificity in a group of 106 non-litigant TBI

patients and a group of 32 healthy controls (Meyers & Volbrecht, 2003).

Suitability of the MNB for Predicting Employment Status

Recent research (Spitz, Ponsford, Rudzki, & Maller, 2012; Williams, Rapport, Hanks,

Millis, & Greene, 2013) has shown that neuropsychological assessment can make unique

contributions to the prediction of employment status after TBI, even after demographic and

neuroimaging data have been considered. The MNB includes subtests that measure each of the

neuropsychological domains listed above that have been found to be significantly related to

employment status following TBI. A brief list of the tests included in the MNB and the

neuropsychological domains that they assess can be found in Table 4. For more complete

descriptions of individual tests, see the Method section below, Lezak et al. (2004), or Strauss,

Sherman, and Spreen (2006).


Table 4

Meyers Neuropsychological Battery Tests Organized by Neuropsychological Domain

Domain MNB Tests

Memory

Rey Auditory Verbal Learning Test

Rey Complex Figure Test (recall and recognition trials)

Digit-Span Backwards (WAIS-III)

Attention

Digit-Span Forward (WAIS-III)

Digit-Symbol Coding (WAIS-III)

Sentence Repetition

Trail Making Test

Visuospatial Skills

Rey Complex Figure Test (copy trial)

Judgment of Line Orientation

Block Design (WAIS-III)

Executive Functions Category Test

Similarities (WAIS-III)

General Cognitive

Functioning

WAIS-III Index Scores (VIQ, PIQ, FSIQ)

Overall test battery mean

Language Fluency Controlled Oral Word Association Test

Animal Naming Test

Motor Finger Tapping Test

Note: WAIS-III = Wechsler Adult Intelligence Scale, Third Edition; VIQ = Verbal

Intelligence Quotient; PIQ = Performance Intelligence Quotient; FSIQ = Full-scale

Intelligence Quotient.


The MNB includes several measures of different memory domains. Among these is the

RAVLT, a well-established verbal word-list learning task similar to the California Verbal

Learning Test-II used by Drake et al. (2000) and Han et al. (2009). The battery also contains the

Rey Complex Figure Test, a test of visual memory. Attention is assessed using the Digit Span

Forward and Digit-Symbol Coding subtests of the WAIS-III, the Sentence Repetition Test, and

the Trail Making Test. Measures of visuospatial skills are provided by the Rey Complex Figure

Test copy trial, Judgment of Line Orientation, and the Block Design subtest of the WAIS-III.

The MNB provides a measure of executive functions using the Category Test, a widely used test

of visual concept formation. The Similarities subtest of the WAIS-III, a test of verbal reasoning

and abstract concept formation, provides another measure of executive functions. General

cognitive functioning is represented by the Performance Intelligence Quotient (PIQ), Verbal

Intelligence Quotient (VIQ), and Full-scale Intelligence Quotient (FSIQ) scores from the Ward

Seven-Subtest Short Form of the WAIS-III, as well as the OTBM as described above. The MNB

contains the COWAT and the Animal Naming Test as measures of language fluency, and the

Finger Tapping Test as a test of motor performance. The battery can also be supplemented with a

number of general mental health questionnaires and measures of demographic variables as

needed.

Predicting Current Employment Status

The literature review presented above outlines a number of studies designed to create

models that predict how long it will take an individual to return to work following TBI (Drake et

al., 2000; Fleming et al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002;

Simpson & Schmitter-Edgecombe, 2002). However, the methodologies used in these studies

only allow for predictions about an individual’s future employment status to be made soon after


a TBI occurs. Furthermore, predictions have only been made about a patient’s employment status

at specific points in time, most commonly at six months or one year after the injury occurs. None

of the studies listed above have investigated the efficacy of neuropsychological assessment

scores to estimate current employment status as a function of time since injury. Such predictions

could be facilitated by assessing individuals at varied time points following a TBI, and

subsequently including the time since injury as a predictor variable in a regression equation used

to predict employment status. Estimates made by this model would be useful in guiding a

clinician’s discussion with their client about whether or not the client is ready to return to work

following TBI, regardless of the amount of time that had passed between the injury and the

assessment. These estimates might also be helpful in identifying neuropsychological domains

that would be appropriate candidates for cognitive intervention. The aim of this study is to create

such a model, in which neuropsychological assessment scores, demographic variables, and time

since injury are used to estimate the current employment status of persons who differ in their

time since TBI.

A study of variables associated with symptoms as well as those who were both referred

and not referred due to litigation. employment status following TBI ideally would involve the

use of cohorts systematically evaluated at designated time points following injury. These

analyses would include those individuals who were both referred and not referred for

neuropsychological evaluations due to persisting cognitive However, the composition of this

sample precludes such an analysis.


Goals

Using scores from a short neuropsychological assessment battery administered between 0

and 55 months after a TBI with a sample of persons referred for neuropsychological assessment,

the goals of the current investigation were to:

1. employ exploratory step-wise regression procedures in order to examine the incremental

proportion of variance in employment status accounted for with the step-wise addition of

each neuropsychological assessment variable, and identify the most parsimonious model

(i.e., the model that includes variables beyond which inclusion of additional variables

does not account for a significant proportion of additional variance).

2. calculate the percentage of cases correctly classified, sensitivity, specificity, positive

predictive value, and negative predictive value in order to examine the extent to which

the regression model mentioned above is able to correctly estimate an individual’s

employment status at the time of assessment.

3. examine the degree to which the addition of the demographic variables of age, premorbid

occupation, education, number of subjective symptoms, ethnic status, and independent

driving status increases the proportion of variance accounted for, as well as the

percentage of cases correctly classified, the sensitivity, the specificity, the positive

predictive value, and the negative predictive value of the regression model described

above.

4. examine the degree to which the addition of neuropsychological variables affects the

proportion of variance accounted for, percentage of cases correctly classified, sensitivity,

specificity, positive predictive value, and negative predictive value of a model which uses

only the demographic variables listed in goal three alone.


5. calculate the proportion of variance accounted for, percentage of cases correctly

classified, sensitivity, specificity, positive predictive value, and negative predictive value

in order to examine the efficacy of the predictive models identified in goals one, three,

and four in an independent and age-matched sample taken from the same database.

6. calculate the proportion of variance accounted for, percentage of cases correctly

classified, sensitivity, specificity, positive predictive value, and negative predictive value

in order to examine the efficacy of the predictive models identified in goals one, three,

and four in two additional samples of litigants and non-litigants taken from the same

database.

7. examine any changes in the proportion of variance in employment status accounted for,

correct classification rate, sensitivity, specificity, positive predictive value, negative

predictive value, or in the predictor variables that account for a significant amount of

variance in employment status when the models mentioned in goals three and four are

created in a sample of 76 litigants taken from the study database.

8. examine any changes in the proportion of variance in employment status accounted for,

correct classification rate, sensitivity, specificity, positive predictive value, negative

predictive value, or in the predictor variables that account for a significant amount of

variance in employment status when the models mentioned in goals three and four are

created using data from all cases contained in the study sample.

The decision to perform analyses based on litigation status in the current sample was based

on consistent findings in prior studies that involvement in litigation can be negatively associated

with performance on neuropsychological assessment measures (Belanger, Curtiss, Demery,

Lebowitz, & Vanderploeg, 2005; Binder & Rohling, 1996; Davis, McHugh, Axelrod, & Hanks,


2012; Meyers, Reinsch-Boothby, & Miller, 2011). Past survey-based research has also shown

that nearly 50% of practicing attorneys agree that they should "always" or "usually" inform their

clients of the presence of effort measures before they participate in a psychological evaluation

(Wetter & Corrigan, 1995), even though such warnings have been shown to alter performance

during neuropsychological assessment (Suhr & Gunstad, 2000). In discussing the assessment of

insufficient effort, a recent consensus conference of the American Academy of Clinical

Neuropsychology also stated that “there is substantial risk that [poor effort] could be present

among the numerous cases assessed by neuropsychologists in a secondary gain setting”, and that

“the presence of problematic effort and response bias can potentially invalidate results”

(Heilbronner, Sweet, Morgan, Larrabee, & Millis, 2009, p. 1121).

It would be possible to conduct further confirmatory analyses by creating additional groups

from the database sample based on important variables such as level of education, ethnic status,

impairment severity, or age. However, due to the limited sample size of this study and the risk of

increased familywise error with multiple analyses, such additional analyses are options for later

exploratory investigations but are beyond the scope of the current study.

Method

Participants

Data were extracted from a private practice neuropsychology database that was collected

in the Midwestern United States. Participants were identified for inclusion in the study if they

carried a diagnosis of TBI and had a complete set of data for all variables of interest. The study

sample included data from 192 participants (130 male and 62 female) between the ages of 18 and

76 (mean = 32.1; SD = 13.1). The study sample consisted of 184 participants who described

themselves as White, 4 participants who described themselves as Hispanic, 3 participants who


described themselves as Native American, and 1 participant who self-identified as being from a

mixed or biracial background.

All participants had been referred for neuropsychological assessment following a TBI as

identified by a primary care provider. Participants in the sample experienced loss of

consciousness ranging from no loss of consciousness to 56 days (mean = 3.6 days; SD = 7.7

days), with 84 participants experiencing loss of consciousness less than 30 minutes, and 108

participants experiencing loss of consciousness of 30 minutes or more. The number of years of

formal education ranged from 6 to 20 years (mean = 12.5 years; SD = 2.0 years). Seventy six

participants in the sample were referred for neuropsychological assessment for purposes related

to litigation, and 116 were referred by physicians to assist in treatment planning. Participants

with missing data values for any of the demographic or neuropsychological assessment variables

of interest were not included in the analysis. Consistent with previous research, (Meyers &

Volbrecht, 2003; Meyers, Volbrecht, Axelrod, & Reinsch-Boothby, 2011) data from participants

who failed two or more embedded effort measures during MNB administration were also

excluded. Embedded effort measures included highly irregular scores on the Rey Complex

Figure Test, Reliable Digit Span, the Forced Choice Test, the Token Test, Dichotic Listening,

Sentence Repetition, the Rey Auditory Verbal Listening Test recognition trial, Estimated Finger

Tapping, and Judgment of Line Orientation.

In order to examine the efficacy of predictive models across subgroups as described in

the goals section, study participants were subdivided in two ways. The first subdivision

randomly divided the sample into two equal-sized groups after the sample had been stratified by

age. The second subdivision was made based on the litigation status of study participants, with

participants who were, and were not, involved in litigation at the time of neuropsychological


assessment being placed in separate groups. These subdivisions were performed independent of

one another, resulting in four nonorthogonal groups as described below.

Group 1. The first group was created by stratifying the 192 study participants by age and

using a random number generator to select half of the study sample for inclusion in Group 1 (n =

96). Although there are additional variables by which the sample could have been stratified, the

size of the study sample limited the feasibility of multiple stratifications. Data from Group 1

were used to construct the initial prediction models, described in Goals 1 through 4 above.

Participants in Group 1 underwent neuropsychological assessment with the MNB

between 0 and 54 months following TBI (mean = 11.04; SD = 14.50). Group 1 consisted of 65

males and 31 females, with ages ranging from 18 to 61 years (mean = 32.40; SD = 12.68). The

number of years of formal education completed by members of Group 1 ranged from 9 to 20

years (mean = 12.54; SD = 1.72). Participants in Group 1 experienced loss of consciousness at

the time of TBI ranging from no loss of consciousness to 28 days (mean = 3.41 days; SD = 7.40

days). SCL-90 global impairment scale T-scores in Group 1 ranged from 40 to 80 (mean = 64.18;

SD = 11.54). Group 1 included 1 participant who self-described as mixed/biracial, 93

participants who described themselves as White, 1 participant who self-described as Native

American, and 1 participant who self-described as Hispanic. Employment backgrounds in Group

1 included 32 participants who were employed in unskilled positions at the time of their injury,

23 participants who were employed in semi-skilled positions, 14 participants who were not

working, 5 participants who were employed in skilled professions, 5 participants who were

working in manager/office/sales positions, 3 participants who were working as

technical/professionals, and 14 participants who were students at the time of their injury. At the

time of neuropsychological assessment, 79 participants in Group 1 were driving independently, 3


were partially driving, and 14 were partially not driving. Forty-nine participants in Group 1 were

involved in litigation at the time of neuropsychological assessment. A summary of demographic

information for Group 1 can be found in Table 5 below.

Group 2. Group 2 consisted of the 96 participants from the study sample who were not

selected for inclusion in Group 1. Data from Group 2 were used to validate the prediction models

created in Group 1. Cross-validation with this sample was done by utilizing a forced-entry

regression model that mimicked the initial prediction models, as described below in Data

Analysis. The purpose of this analysis was to investigate the efficacy of the initial prediction

models in an independent sample as described in Goal 5 above.





years (mean = 12.53; SD = 2.24). Participants in Group 2 experienced loss of consciousness at

the time of TBI ranging from no loss of consciousness to 56 days (mean = 3.85 days; SD = 8.07


SD = 12.18). Group 2 included 91 participants who described themselves as White, 2

participants who described themselves as Native American, and 3 participants who described

themselves as Hispanic. Employment backgrounds in Group 2 included 26 participants who were

employed in unskilled positions at the time of their injury, 15 participants who were employed in

semi-skilled positions, 14 participants who were not working, 6 participants who were employed

in skilled professions, 11 participants who were working in manager/office/sales positions, 6

participants who were working as technical/professionals, and 18 participants who were students


at the time of their injury. At the time of neuropsychological assessment, 74 participants in

Group 2 were driving independently, 5 were partially driving, and 17 were partially not driving.

Thirty-five participants in Group 2 were involved in litigation at the time of neuropsychological

assessment. A summary of demographic information for Group 2 can be found in Table 5 below.

Group 3. The third group was composed of 74 participants from the study sample who

were involved in litigation directly related to their head injury at the time of neuropsychological

assessment. As mentioned above in Goals 6 and 7, Group 3 was created to examine the degree to

which litigation status was associated with neuropsychological test performance and employment

following a TBI.





years (mean = 12.61; SD = 1.95). At the time of TBI, participants in Group 3 experienced loss of



SD = 10.12). Group 3 included 1 participant who self-described as mixed/biracial, 73

participants who described themselves as White, and 2 participants who described themselves as

Native American. Employment backgrounds in Group 3 included 26 participants who were







Group 3 were driving independently, 2 were partially driving, and 8 were partially not driving. A

summary of demographic information for Group 2 can be found in Table 5 below.

Group 4. The fourth group was composed of 115 participants from the study sample who

were not involved in litigation directly related to their head injury at the time of

neuropsychological assessment As mentioned above under Goals 6 and 7, Group 4 was created

to make non-statistical comparisons with Group 3 based on the results of exploratory stepwise

logistical regression models, to examine the degree to which litigation status was associated with

neuropsychological test performance and employment following a TBI.





years (mean = 12.45; SD = 1.99). At the time of TBI, participants in Group 4 experienced loss of



SD = 12.54). Group 4 included 110 participants who described themselves as White, 4

participants who described themselves as Hispanic, and 1 participant who described themselves

as Native American. Employment backgrounds in Group 4 included 32 participants who were







Group 4 were driving independently, 6 were partially driving, and 23 were partially not driving.

A summary of demographic information for Group 4 can be found in Table 5 below. Results of

MANOVA analysis were non-significant (λ = 0.96, p = 0.76), indicating that differences

between groups 1, 2, 3, and 4 on the demographic variables of interest did not exceed those

expected by chance.


Table 5

Demographic Variables of Interest in Groups 1-4 and the Full Study Sample

Group 1 Group 2 Group 3 Group 4 Full Sample

Variable Mean SD Mean SD Mean SD Mean SD Mean SD

Age 32.4 12.7 31.8 13.6 34.1 12.8 30.7 13.2 32.1 13.1

Education 12.5 1.7 12.5 2.2 12.6 1.9 12.5 2.0 12.5 2.0

SCL-90 64.2 11.5 64.2 12.2 67.1 10.1 62.4 12.5 64.2 11.8

Loss of Consciousness 3.4 7.4 3.8 8.1 2.0 4.8 4.7 8.9 3.6 7.7

Variable n % n % n % n % n %

Sex

Male 65 68% 65 68% 53 70% 76 66% 130 68%

Female 31 32% 31 32% 23 30% 39 34% 62 32%

Premorbid Occupation

Not Working 14 15% 14 15% 7 9% 21 18% 28 15%

Semi-skilled 55 57% 41 43% 42 55% 54 47% 96 50%

Skilled 27 28% 41 43% 27 36% 40 35% 68 35%

Driving Status

Driving 82 85% 79 82% 68 89% 92 80% 161 84%

Not Driving 14 15% 17 18% 8 11% 23 20% 31 16%

Note: Results of MANOVA showed no significant between-group differences on variables of

interest. Cumulative percentages for some demographic variables may add up to more than

100% after rounding. Loss of consciousness is measured in days. SCL-90 = Symptom

Checklist-90 Global Impairment T-score.


Procedure

Neuropsychological examination. All participants were referrals to a private practice

neuropsychology clinic located in the Midwestern United States. Referrals were made as part of

routine neuropsychological practice, including referrals from neurologists and physicians to

assist in treatment planning. Each participant underwent neuropsychological assessment using

the MNB between 0 and 55 months following a TBI. All testing was performed during a single

office visit by a board certified clinical neuropsychologist or a trained psychometrist under the

supervision of the same neuropsychologist. Each visit also included a semi-structured interview

conducted by the neuropsychologist. All testing was conducted in an office space specifically

designated for neuropsychological test administration, and scores were entered into a

computerized database at the time of assessment.

Administration of the battery followed standardized MNB procedures, which include the

following subtests in the order presented: the Ward Seven Subtest form of the WAIS-III, a

forced-choice effort measure, Rey Complex Figure Test copy trial, Animal Naming, One Minute

Estimation, Rey Complex Figure Test immediate recall, COWAT, Dichotic Listening, the North

American Adult Reading Test, Sentence Repetition, Rey Complex Figure Test delay recall, Rey

Complex Figure Test recognition trial, RAVLT acquisition, distraction list, and immediate recall

trials, Judgment of Line Orientation, Boston Naming Test, Finger Tapping, Finger Localization,

Trail Making Test parts A and B, RAVLT delayed recall and recognition trials, Token Test, and

the Category Test. During administration, participants were offered a short break following the

WAIS-III, the Rey Complex Figure Test recognition trial, and the Category Test.

Following the Category Test, each participant was asked to complete the Symptom

Checklist-90-R (Derogatis, 1994). Participants also participated in a brief semi-structured


interview with the board certified clinical neuropsychologist designed to collect demographic

and background information, including employment status, sex, age in years, years of education

completed, ethnicity, handedness, marital status, and occupation. Data regarding whether or not

each participant was driving independently was also collected during the interview as an

indicator of independent functioning. The number of months since injury at the time of

neuropsychological assessment was also recorded. The variable number of months since injury

was grouped with neuropsychological assessment variables rather than demographic variables in

the current study so that it would be included in all planned analyses.

Assessment domains, instruments and measures. The neuropsychological measures

included in the data analyses represent the neuropsychological domains previously shown to be

significantly related to employment status following TBI. To the extent possible,

neuropsychological domains were measured using the same tests that have been found in

previous studies to be significantly related to employment status following TBI.

Memory.

RAVALT. The RAVLT was used to obtain measures of immediate, delayed, and

recognition memory. The RAVLT is a well-established verbal word-list learning task in which

the participant is asked to memorize a list of 15 words over five learning trials. The

memorization is followed by a distractor list trial, immediate and 20-minute delayed recall trials,

and a recognition trial. The score for each trial is the number of words correctly recalled or

recognized. The numerous recall trials allow the test to assess short term, long term, and

recognition verbal memory.

Factor analysis and structural equation modeling studies have shown that the RAVLT can

be described by two-factor models that represent either acquisition and retention (Vakil &


Blachstein, 1993) or short-term memory and long-term memory (Muller, Hasse-Sander, Horn,

Helmstaedter, & Elger, 1997). Factor analytic studies have also shown the RAVLT to be a test of

verbal learning and memory that are distinct from measures of attention, concentration and

intelligence (Ryan, Rosenberg, & Mittenberg, 1984). Scores from the RAVLT have been found

to have moderate convergent validity with scores from other measures of verbal memory,

including correlations ranging from r = 0.50 to r = 0.58 with the Wechsler Memory Scale-

Revised Logical Memory subtest and correlations ranging from r = 0.49 to r = 0.83 with the

California Verbal Learning Test (Delis, Kramer, Kaplan, & Ober, 1987; Johnstone, Vieth,

Johnson, & Shaw, 2000; Stallings, Boake, & Sherer, 1995). Scores from the RAVLT have also

been found to have modest correlations (r = 0.37 – r = 0.44) with scores from the Benton Visual

Retention Test, a test of visual memory (Magalhaes, Malloy-Diniz, & Hamdan, 2012). Scores

from the RAVLT have demonstrated good divergent validity (i.e. low correlation coefficients

ranging from r = 0.01 to r = 0.22) with scores from the Trail Making Test, a test of visual

attention (Magalhaes, Malloy-Diniz, & Hamdan, 2012). It has been found to be a valid measure

of left-hemisphere temporal lobe damage. Ivnik, Sharbrough, and Laws (1988) found that scores

from each of the learning trials, the immediate recall trial, and the delayed recall trial all

significantly discriminated (p < 0.01) between patients with intractable epilepsy who had

undergone temporal lobectomy on either the right or left side, with left-side lobectomy resulting

in lower scores. Similarly, Miceli, Caltagirone, Gainotti, Masullo, and Silveri (1981) found that

non-aphasic patients with left-hemisphere brain lesions scored significantly (p < 0.05) worse

than non-aphasic patients with right-hemisphere brain lesions on both the immediate and delayed

recall trials of the RAVLT.


Scores from the RAVLT have also demonstrated moderate test-retest reliability over

periods of 14-35 days, with Pearson r values ranging from 0.68 to 0.78 for the trials 1-5 total

score, 0.68 to 0.76 for the immediate recall trial, and 0.71 to 0.81 for the delayed recall trial

(Lemay, Bedard, Rouleau, & Tremblay, 2004; Magalhaes, Malloy-Diniz, & Hamdan, 2012). At

an interval of three months, test-retest reliability was found to be r = 0.70 for the trials 1-5 total

score, and r = 0.62 for the delayed recall trial (van den Burg & Kingma, 1999). The test has also

been found to have high internal consistency across learning trials as measured by Cronbach’s

coefficient α, with scores for different forms ranging from 0.80 to 0.91 (Magalhaes, Malloy-

Diniz, & Hamdan, 2012; van den Burg & Kingma, 1999).

Rey Complex Figure Test. The Rey Complex Figure Test was used to obtain measures of

visuospatial processing and visual memory. Administration and scoring of the Rey Complex

Figure Test followed the guidelines presented by Meyers and Meyers (1995). During

administration of the Rey Complex Figure Test, the participant is presented with a printed

reproduction of a complicated geometric figure, a pencil, and a blank piece of paper. They are

then instructed to copy the figure to the blank piece of paper and encouraged to make their

drawing as accurate as possible. The test also consists of two recall trials in which the participant

is asked to reproduce the figure from memory, an immediate recall trial that takes place 3

minutes after completing the copy trial and a delayed recall trial that takes place 30 minutes after

completing the copy trial. Following the delayed recall trial the patient participates in a

recognition trial. During the recognition trial, 12 individual elements of the original complicated

figure are presented alongside 12 distractor designs, and the patient is asked to circle all of the

elements that they recognize as part of the original figure.


For the copy, immediate recall, and delayed recall trials, the figure is divided into 18

distinct elements, and each element is assigned a score of 0 (not present), 0.5 (misplaced and

inaccurately drawn), 1 (misplaced but accurately drawn or properly placed but inaccurately

drawn), or 2 (properly placed and accurately drawn), resulting in a total possible score of 36

points. The recognition trial is scored by calculating the number of true positive, false positive,

true negative, and false negative responses. A total recognition score is also obtained by adding

the number of true positives and the number of true negatives.

A number of factor analytic studies have demonstrated that variability in scores from the

Rey Complex Figure Test often demonstrate significant convergent validity with scores from

other tests of memory and perceptual organization, while at the same time showing good

divergent validity with scores from tests designed to measure unrelated constructs. In a sample of

260 adults with head injuries, scores from the copy and immediate recall trials of the Rey

Complex Figure Test shared stronger correlations with other measures of perceptual organization

(r = 0.35 for copy trial and r = 0.50 for immediate recall trial), than they did with measures of

verbal comprehension (r = 0.10 for copy trial and r = 0.16 for immediate recall trial) or of

freedom from distractibility (r = 0.18 for copy trial and r = 0.19 for immediate recall trial;

Sherman, Strauss, Spellacy, & Hunter, 1995). In a separate factor analysis of memory

performance scores, measures obtained from the Rey Complex Figure Test immediate and

delayed recall trials loaded onto a nonverbal memory factor (r = 0.88 and r = 0.85, respectively)

along with measures from the Visual Reproduction subtest of the Wechsler Memory Scale

(Wechsler, 1945), another commonly used test of visual memory (Ostrosky-Solis, Jaime, &

Ardila, 1998). Finally, in a recent factor analysis of neuropsychological battery scores conducted

by Ponton, Gonzalez, Hernandez, Herrera, & Higareda (2000), a visuospatial factor emerged


which consisted of the Rey Complex Figure Test copy trial (r = 0.80) and immediate recall trial

(r = 0.84), as well as Raven’s Standard Progressive Matrices (r = 0.51; Raven, Raven, & Court,

1993), a test of visuospatial reasoning.

The manual for the Rey Complex Figure Test Meyers and Meyers (1995) scoring system

reports interrater reliabilities ranging from r = 0.93 and r = 0.99 for the copy, immediate recall,

and delayed recall trials, with a median interrater reliability coefficient of r = 0.94. The manual

also reports test-retest reliability coefficients of r = 0.76 for the immediate recall trial, r = 0.89

for the delayed recall trial, and r = 0.87 for the recognition total correct trial. Test-retest

reliabilities for the copy trial were not reported due to the restricted range of patient scores on

this particular subtest, which are usually near perfect. Other research (Taylor, Leung, & Deane,

2011) has suggested that scores from the copy trial of the Rey Complex Figure Test show modest

correlations with functional outcomes following TBI, specifically the ability to drive

independently.

Attention.

Digit-Symbol Coding. The WAIS-III Digit-Symbol Coding test was used to obtain a

measure of attention. During administration of the Digit-Symbol Coding subtest of the WAIS-III,

the patient is asked to pair written symbols with numbers using a key presented at the top of the

page. The test produces a single score, which is the number of symbols correctly copied by the

patient with a 120-second time period. Successful completion of the test depends on motor

persistence, sustained attention, response speed, and visuomotor coordination rather than on

memory or learning (Lezak et al., 2004). Mertens, Gagnon, Coulombe, and Messier (2006) found

that across age groups, scores from the Digit-Symbol Coding subtest correlated more strongly

with scores from the Symbol Search subtest of the WAIS-III, another test of visual-motor


attention (r = 0.83), than other subtests (r = 0.27 – 0.48). Ward, Ryan, and Axelrod (2000) used

exploratory factor analysis in the WAIS-III standardization sample to show that in a three-factor

model, scores from Digit-Symbol Coding shared the most variance with scores from other

subtests of attention and working memory such as Arithmetic, Digit-Span, and Letter-Number

Sequencing. In a sample of participants with moderate to severe TBI, scores from the Digit-

Symbol Coding subtest have shown high convergent validity with scores from the Repeatable

Battery for the Assessment of Neuropsychological Status Coding subtest (Randolph, 1998; r =

0.83), a similar test of visual-motor attention (McKay, Casey, Wertheimer, & Fichtenberg,

2007). Scores on the Digit-Symbol Coding subtest have been found to be sensitive to brain

injury, with TBI patients scoring significantly (p < 0.01) lower than age-, gender-, and education-

matched healthy controls, and with Cohen’s d effect sizes ranging from -0.61 to -1.47 depending

on the severity of injury (Langeluddecke & Lucas, 2003). Finally, a functional MRI study

performed by Usui and colleagues (2009) found that performance on the WAIS-III Digit-Symbol

Coding subtest was related to activation of prefrontal cortical areas involved in sustained

attention and performance on tasks of working memory.

Trail Making Test, Part B. The Trail Making Test, Part B was used to obtain a second

measure of attention. The Trail Making Test is a test of visuospatial attention that is conducted in

two parts. During administration of the first part of the test (Part A), the patient is presented with

a sheet of paper containing randomly-placed circles numbered from 1 to 25 and is asked to

connect the circles in numerical order as quickly as possible. Part A is thought to provide a

measure of scanning and visuomotor tracking. During the second part of the test (Part B), the

patient is asked to draw a line connecting numbered and lettered circles both numerically and

alphabetically, alternating between the numbered circles and the lettered circles. Part B is


thought to provide a measure of divided attention and cognitive flexibility. Moderate test-retest

reliabilities have been noted over a nine-month period for both Part A (r = 0.79) and Part B (r =

0.89; Mitrushina, Boone, Razani, & D’Elia, 2005). However, test-retest reliabilities are generally

higher for Part A than for Part B (Bornstein, Baker, & Douglas, 1987; Dikmen, Heaton, Grant, &

Tempkin, 1999; Mitrushina & Satz, 1991). A moderate correlation has been noted between

scores from the Trail Making Test Parts A and B (r = .31; Strauss et al., 2006), supporting the

idea that the two tests measure similar but nonequivalent constructs. The Trail Making Test has

also been found to have moderate to strong positive correlations with other tests of attention,

such as the Symbol-Digit Modalities Test (r = 0.39), the Visual Search and Attention Test (r =

0.50), and the Paced Serial Addition Test (r = 0.60; O’Donnell, McGregor, Dabrowski,

Oestreicher, & Romero, 1994; Royan, Tombaugh, Rees, & Francis, 2004).

Acker and Davis (1989) demonstrated the efficacy of the Trail Making Test in predicting

functional outcomes following TBI by using the Social Status Outcome Survey (Cope, 1982) to

track functional outcomes of 148 participants who had undergone neuropsychological

assessment following a TBI. Participants completed the Social Status Outcome Survey an

average of 3.8 years following assessment. Results showed significant correlations between

Social Status Outcome Survey and the Trail Making Test part A (r = 0.31) and Part B (r = 0.33).

Similar correlations with other measures of functional outcome have been noted elsewhere

(Ross, Millis, & Rosenthal, 1997).

Visuospatial skills.

Judgment of Line Orientation. The Judgment of Line Orientation test was used to obtain

a measure of visuospatial skills. Judgment of Line Orientation is a popular test of visual

perception in which a patient is shown 11 numbered radii that form a semicircle. For each test


item, the patient is presented with two additional angled radii and asked to identify the numbered

radii that are oriented identically to the item stimulus. Benton, Varney, and Hamsher (1978)

report high split-half reliabilities ranging from 0.89 to 0.94, as well as a test-retest reliability of

0.90 over a period of up to 21 days. Spencer, Wendell, Giggey, Seliger, Katzel, and Waldstein

(2013) found that the Judgment of Line Orientation test showed high internal consistency in both

a sample of 128 undergraduate students (α = 0.84) and 203 stroke- and dementia-free,

community dwelling older adults with a mean age of 66 years (α = 0.81). As a test of

visuospatial processing, Judgment of Line Orientation is widely considered to be sensitive to

right-hemisphere lesions (Lezak et al., 2003; Mitrushina et al., 2005), although this claim has

recently been called into question (Treccani & Cubelli, 2011).

Block Design. The Block Design subtest of the WAIS-III was used to obtain an additional

measure of visuospatial skills. During administration of the Block Design subtest of the WAIS-

III, the patient is asked to use between two and nine red and white colored blocks to recreate

patterns presented in a stimulus book. Similar to other construction tasks, the test measures both

spatial perception as well as motor execution (Lezak et al., 2004). Factor analysis showed that

the Block Design subtest was most strongly correlated with other WAIS-III subtests measuring

visuospatial constructs such as Matrix Reasoning, Visual Puzzles, and Picture Completion (The

Psychological Corporation, 2002). The test has been shown to be sensitive to closed-head

injuries in both hemispheres, but especially in the right hemisphere (Wilde, Boake, & Sherer,

2000).

Scores from the Block Design subtest have shown split half reliability coefficients of r =

0.85 to r = 0.90 in individuals under 75 years of age (The Psychological Corporation, 2002), and

a split half reliability of r = 0.92 in a sample of 22 individuals with TBI (Zhu, Tulsky, Price, &


Chen, 2001). Using data from the WAIS-III standardization sample, Iverson (2001) categorized

the Block Design subtest as one of the “most reliable” subtests of the WAIS-III, meaning that

scores from the test demonstrated internal consistency ranging from r = 0.88 to r = 0.99, as well

as test-retest reliability ranging from r = 0.75 to r = 0.99 (p. 186).

Executive functions.

Category Test. The Victoria Revision of the Category Test was used to obtain a measure

of executive functions. The Category Test is a test of visual concept formation, and was one of

the seven tests originally included in the Halstead Neuropsychological Battery. The test contains

seven subtests in which different patterns are presented visually, and the patient is told that each

pattern will remind him or her of a number between one and four. For each subtest, there is a

consistent principle that determines which number is represented, and it is up to the patient to

discover the principle and apply it to each item. The patient is able to test different principles by

using feedback from the examiner, who tells him or her whether each of their answers is correct

or incorrect. Scores on the Category Test show moderate positive correlations with Wisconsin

Card Sort Test, another popular test of visual concept formation (Strauss et al., 2006). While the

Category Test and the Wisconsin Card Sort Test are both thought to be tests of visual concept

formation, the modest correlations between the two are most likely due to the different formats

of the two tests. Adams and Trenton (1981) report that the Category Test is almost as valid as the

complete Halstead Neuropsychological Battery in detecting the presence or absence of brain

damage, citing the fact that 95% of individuals with brain damage scored significantly below

matched controls.

Using coefficient α, Lopez, Charter, and Newman (2000) found high internal consistency

for the Category Test total score (r = 0.97), as well as for subtests III to VII (r = 0.77 - 0.96).


The Victoria revision of the Category Test is a condensed version that only contains 86 of

the original 208 test items. In a sample of 86 adults with closed head injuries resulting from

motor vehicle accidents, scores did not differ significantly between 43 individuals who were

assessed using the Victoria revision (mean score = 41.88, SD = 13.03) and 43 individuals were

given the full Category Test (mean score = 41.79, SD = 10.34; Kozel & Meyers, 1998).

Similarities. The Similarities subtest of the WAIS-III was used to obtain an additional

measure of executive functions. The Similarities subtest is an instrument that provides a measure

of verbal reasoning and abstract concept formation. In it, the patient is presented with two words

and asked to identify how the words are similar to one another. The word pairs presented initially

represent very concrete relationships, but later progress to represent relationships which are

much more abstract. More points are awarded for answers that identify an abstract relationship

rather than a concrete likeness. Van der Heijden and Donders (2003) administered the WAIS-III

to 166 participants with TBI who were consecutive referrals to a Midwestern rehabilitation

facility. Confirmatory factor analysis supported a four-factor model for the WAIS-III, where the

Similarities subtest shared more variance with tests of verbal comprehension (p = 0.83) than with

tests of perceptual organization, working memory, or processing speed. The test has also been

found to have high split-half reliability in a sample of 22 TBI patients (r = 0.96; Zhu et al.,

2001). Scores on the Similarities subtest have been found to be sensitive to brain injury, with

TBI patients scoring significantly (p < 0.01) lower than age-, gender-, and education-matched

healthy controls, with Cohen’s d effect sizes ranging from -0.38 to -1.02 depending on the

severity of injury (Langeluddecke & Lucas, 2003).


General cognitive function.

WAIS-III index scores. The PIQ, VIQ, and FSIQ scores from the Ward Seven-Subtest

Short Form of the WAIS-III were used to obtain measures of general cognitive function. The

Ward Seven-subtest Short Form of the WAIS-III is a condensed administration of the traditional

WAIS-III that only includes the Information, Digit Span, Arithmetic, Similarities, Picture

Completion, Block Design, and Digit-symbol Coding subtests. Just like its full-form counterpart,

the Ward short form produces PIQ, VIQ and FSIQ scores. Pilgrim, Meyers, Bayless, and

Whetstone (1999) have shown that these short form index scores correlate strongly with the same

index scores obtained during full-form WAIS-III administration. They calculated and compared

PIQ, VIQ, and FSIQ scores from both the full-form and Ward Seven-subtest Short Form of the

WAIS-III in a sample of 111 individuals from a semirural area. They found that scores were

strongly correlated for the PIQ (r = 0.95), VIQ (r = 0.97), and FSIQ (r = 0.98), demonstrating the

usefulness of the short form as a valid measure of general cognitive functioning.

Overall test battery mean. The MNB’s OTBM score was used to obtain another measure

of general cognitive function. An OTBM is a descriptive statistic used to summarize

performance on a complete neuropsychological battery. The score is derived by calculating the

mean of the T-scores from each test included in the battery. A detailed description of the process

used for calculating the OTBM, as well as a discussion of its clinical utility has been provided by

Miller and Rohling (2001). The OTBM for each participant in the current study was calculated

using T-scores from all MNB subtests.

Language fluency.

COWAT. The COWAT was used to obtain a measure of language fluency. The COWAT

is a well-established test of phonemic fluency that has been shown to be a sensitive indicator of


brain dysfunction (Strauss et al., 2006). During COWAT administration, the patient is assigned a

letter of the alphabet and told to verbally list as many words as possible beginning with that letter

within a 60 second time limit. Patients are told to specifically avoid proper names as well as

different forms of the same word (i.e. ‘eat’ and ‘eating’). The process is repeated a total of three

times with different letters of the alphabet, and a total score is derived. Correlations between

tests of phonemic fluency using different letters of the alphabet have been shown to be high (r =

0.86; Ross, Furr, Carter, & Weinberg, 2006). The COWAT has also shown moderate correlations

(r = 0.52) with the Animal Naming Test, a measure of semantic fluency (Tombaugh, Kozak, &

Rees, 1999). It has also shown correlations ranging from r = 0.64 to r = 0.87 with the WAIS-III

Verbal Intelligence Quotient Index score, suggesting a strong verbal component to test

performance (Henry & Crawford, 2004).

Ruff, Light, Parker, and Levin (1996) found a high coefficient α for the three letters

presented during COWAT administration (r = 0.83), indicating high internal consistency across

items. They also noted a test-retest reliability of r = 0.74 over a period of six months. Interrater

reliabilities for the COWAT have been described as “near perfect” (Mitrushina et al., 2005, p.

202), and have been reported to be as high as r = 0.98 (Norris, Blankenship-Reuter, Snow-Turek,

& Finch, 1995).

Motor performance.

Finger Tapping Test. The Finger Tapping Test was used to obtain a measure of motor

performance. The Finger Tapping Test is a test of motor speed in which the patient is presented

with a flat wooden board to which a mechanical counter with a small lever is attached. The

patient is asked to rest their hand on the board and tap the lever with their index finger as quickly

as possible for a period of ten seconds. The mechanical counter keeps track of the number of


times the patient taps the lever during each trial. The patient repeats the process between five and

ten times with each hand, depending on the consistency of scores across trials. The Finger

Tapping Test was one of the tests used by Ward Halstead in his neuropsychological assessment

battery and is one of the most widely used tests of manual dexterity (Lezak et al., 2004).

The Finger Tapping Test has shown high convergent validity across handedness

asymmetries with the Purdue Peg Placement Test (Tiffin, 1968), a measure of motor dexterity (r

= 0.78; Triggs, Calvanio, Levine, Heaton, & Heilman, 2000). The Finger Tapping Test has also

shown low correlation with the Processing Speed Index of the WAIS-III (r = 0.25), suggesting

that it is able to measure speed of motor performance independent from speed of information

processing (Kennedy, Clement, & Curtiss, 2003). Scores on the Finger Tapping Test have been

found to correlate significantly (p = 0.01) with damage to the splenium of the corpus callosum 6-

months after moderate to severe TBI (Farbota, Bendlin, Alexander, Rowley, Dempsey, &

Johnson, 2012). Finger Tapping scores have also been found to share moderate correlations with

a number of functional outcomes following brain injury, including employment outcomes

(Prigatano, 1999).

Data Reduction

Demographic Variables.

In several cases, demographic variables were coded into multiple categories during data

collection. After coding, the demographic variables employment status, ethnicity, premorbid

occupation, and independent driving status demonstrated low membership in certain categories.

Such cases of low expected frequencies can result in decreased statistical power and inflated

odds ratios (Tabachnick & Fidell, 2007). In order to maximize statistical power, these variables

were re-coded into fewer categories in order to increase membership in each individual category.


Employment status. At the time of neuropsychological assessment, the employment

status of each participant was coded into one of nine possible categories. The coding included

seven categories for use with civilians and two categories for use with active duty members of

the armed forces. For civilians the categories were retired, disabled, not employed, volunteer,

non-competitive, below premorbid (i.e. a position that was considered less competitive than

premorbid employment), and same as premorbid (i.e. employment that was considered equally

competitive as premorbid employment). For active duty military the categories were

deployable/full duty, and non-deployable.

For the purposes of the current study, these employment categories were originally

collapsed into three main employment outcome groups based on the estimated

neuropsychological demand of each category. The three employment outcome groups were not

employed, partially employed, or fully employed. The employment outcome group not employed

included those who were rated as retired, disabled, and not employed. The employment outcome

group partially employed included those who were rated as volunteer, non-competitive, below

premorbid, and non-deployable. The employment outcome category fully employed included

those who were rated same as premorbid and deployable/full duty. However, under the

conditions described above, only one participant from the sample qualified as fully employed,

making the category impractical for use in data analysis. Instead, the employment categories

partially employed and fully employed were collapsed into a single group. The result was two

employment categories, not employed and employed, which were used as the criterion measure

for binary logistic regression analyses.

Ethnicity. During the neuropsychological assessment, the self-reported ethnicity of each

participant was coded into one of seven categories: African American, Asian, White, Hispanic,


Native American, Pacific Islander, or Mixed/Biracial. However, the ethnic composition of the

sample, which was predominantly White, resulted in low membership in ethnic minority

categories. In order to increase group membership and thus increase statistical power, the

ethnicity of each participant was re-coded into one of two categories: White or Ethnic Minority.

The category Ethnic Minority included the six minority categories that had previously been

coded individually.

Even after re-coding the ethnicity of each participant in the database sample, membership

in the Ethnic Minority category was low (Group 1 n = 3; Group 2 n = 5; Group 3 n = 3; Group 4

n = 5). Low group membership in the Ethnic Minority category resulted in reduced statistical

power and odds ratios which were difficult to interpret (Tabachnick & Fidell, 2007). The

variable ethnicity was therefore removed from data analysis and not considered in the creation of

any of the seven regression models.

Premorbid Occupation. At the time of neuropsychological assessment, the occupation of

each participant was recorded and coded into one of seven categories: technical/professional,

manager/office/sales, skilled, student, semi-skilled, unskilled, and, not working. In order to

maximize statistical power, employment categories were condensed into three groups for

analysis: not working (Group 1 n = 14; Group 2 n = 14; Group 3 n = 7; Group 4 n = 21), semi-

skilled (Group 1 n = 55; Group 2 n = 41; Group 3 n = 42; Group 4 n = 54), and skilled (Group 1

n = 27; Group 2 n = 41; Group 3 n = 27; Group 4 n = 40). The semi-skilled group contained the

categories unskilled and semi-skilled, while the skilled group contained the categories skilled,

manager/office/sales, technical/professional, and student.

Independent driving status. At the time of neuropsychological assessment, the driving

status of each participant was coded as driving, partially driving, partially not driving, or not


driving. In order to maximize statistical power, these categories were combined into two levels:

driving, which consisted of the prior categories driving and partially driving (Group 1 n = 82;

Group 2 n = 79; Group 3 n = 68; Group 4 n = 92), and not driving, which consisted of the prior

categories partially not driving and not driving (Group 1 n = 14; Group 2 n = 17; Group 3 n = 8;

Group 4 n = 23).

Neuropsychological assessment measures.

The variable percent change in RAVLT scores between short-delay and long-delay recall

trials was originally planned for inclusion in data analysis, based on evidence provided by Han et

al. (2009) that a similar score taken from the California Verbal Learning Test, Second Edition

(Delis et al., 2000) significantly distinguished between groups of employed and unemployed

individuals following TBI. However, calculation of this score became problematic for

participants who did not recall any words during the short-delay free recall trial of the RAVLT.

Han and colleagues calculated their percent change score by using the following formula:

(long delay free recall score – short delay free recall score) / (short delay free recall score)

For participants who did not recall any words during the short delay recall trial, the formula

required dividing by zero and was therefore unsolvable. This created missing values for 4

participants in Group 1 and 5 participants in Group 2, thus precluding these data from analysis.

After original analyses were conducted, it was noted that the percent change variable had not

reached statistical significance in any of the regression models. In order to maximize sample size

and statistical power, the variable was removed from analysis and all models were recalculated.

In an attempt to minimize the number of predictor variables, specific Symptom

Checklist-90-R subscale scores were not considered for inclusion in analyses. This is consistent

with past research demonstrating that TBI-related symptom endorsements on the Symptom


Checklist-90-R are no more predictive of functional outcomes following TBI than non-TBI-

related symptom endorsements (Hoofien, Barak, Vakil, & Gilboa, 2005).

Data Analysis

All analyses were performed using SPSS version 20. Predictor variables were tested for

any violation of linearity in the logit by using a Box-Tidwell approach in which each variable is

entered into a regression model along with an interaction term for the variable and its natural

logarithm. Violations of linearity in the logit are detected when interaction terms make

significant contributions to the final model (Tabachnick & Fidell, 2007). Forward stepwise

binary logistic regression analysis was then used to create eight models in order to accomplish

the eight study goals outlined in the Goals section above. For a summary of all eight models

created during analysis and the relevant groups in which the models were created and confirmed,

see Table 6.


Table 6

Exploratory Stepwise Binary Logistic Regression Models Created

Model

Number

Data

Source

Primary

Predictors

Secondary

Predictors

Criterion Variable Confirmation

Group(s)

1 Group 1

Neuropsychological

Assessment

Variables

None Employment Status at

time of assessment None

1b Group 1

Neuropsychological

Assessment

Variables

None Employment Status at

time of assessment Groups 2, 3, 4

2 Group 1

Neuropsychological

Assessment

Variables

Demographic Variables Employment Status at

time of assessment Groups 2, 3, 4

3 Group 1 Demographic

Variables

Neuropsychological

Assessment Variables

Employment Status at

time of assessment Groups 2, 3 4

4 Group 3

Neuropsychological

Assessment

Variables


time of assessment Group 3

5 Group 3 Demographic

Variables

Neuropsychological



time of assessment Group 3

6 Entire

Sample

Neuropsychological

Assessment

Variables


time of assessment Entire Sample

7 Entire

Sample

Demographic

Variables

Neuropsychological



time of assessment Entire Sample

Note: Data Source = The group from which data was utilized in the creation of the model;

Primary Predictors = variables entered into the regression model in Block 1; Secondary

Predictors = variables entered into the regression model in Block 2; Criterion Variable =

criterion variable that was used in the creation of the model; Confirmation Group = the group in

which the fit of the model was tested.


Goal 1. Employ exploratory step-wise regression procedures in order to examine the

incremental proportion of variance in employment status accounted for with the step-wise

addition of each neuropsychological assessment variable, and identify the most parsimonious

model (i.e., the model that includes variables beyond which inclusion of additional variables


In order to accomplish Goal 1, Model 1 was created using data from participants in

Group 1. Neuropsychological test scores from the MNB were used as predictor variables, while

the employment status of each participant at the time of neuropsychological assessment was used

as a dichotomous criterion measure. A complete listing of the neuropsychological assessment

measures used in analysis can be found in Table 7. While additional predictor variables could

have been chosen from the MNB, the study sample size and statistical power considerations

precluded their inclusion. The MNB measures listed in Table 7 are those with the strongest

empirical support as predictors of employment outcomes following TBI.

All neuropsychological assessment predictor measures were entered into binary logistic

regression analysis in a single block using a forward conditional stepwise entry method. Default

settings in SPSS were changed to include a probability for stepwise entry of 0.15 and a

probability of stepwise removal of 0.40. The use of less stringent inclusion criteria in exploratory

logistic regression techniques was recommended by Tabachnick and Fidell (2007) to maximize

predictive accuracy by ensuring the inclusion of variables with beta coefficients that are different

from zero. The use of a binary logistic regression analysis was necessary in the creation of all

models due to the reduction of the planned criterion variable employment status to a dichotomous

variable, as described above under Data Reduction.


The adjusted R2 value was calculated using SPSS software during model creation as a

measure of the amount of variance in employment status that was accounted for by Model 1. Due

to the use of binary logistic regression analysis, the significance of the χ2 change for each step

was also calculated using SPSS software as a way of determining whether the addition of each

new variable accounted for a significant amount of variance in the criterion variable.

In order to examine the aggregated effect of including additional neuropsychological

variables, one additional model, to be known as Model 1b, was created by repeating the analyses

described above and allowing variable inclusion to continue until the specified selection criteria

did not result in the inclusion of any additional variables. Model creation was allowed to

continue regardless of the significance of the χ2 change statistic for each step. Due to the

hypothesized robustness of this new model, which would theoretically account for a larger

amount of variance in employment status in Group 1, it was planned for use in all subsequent

analyses.

Goal 2. Calculate the percentage of cases correctly classified, sensitivity, specificity,

positive predictive value, and negative predictive value in order to examine the extent to which

the regression model mentioned above is able to correctly estimate an individual’s employment

status at the time of assessment.

In order to accomplish Goal 2, employment status predictions were created for each

participant in the study sample using Model 1b following the initial step-wise, forward entry

regression analysis. The decision to use Model 1b for confirmatory analyses was based on that

model’s inclusion of more predictor variables, making it more robust than Model 1, which

contained only one predictor variable and which accounted for a significantly smaller proportion

of variance in employment status. Predicted employment status was then compared to observed


employment status in order to classify each individual prediction as a true positive, a false

positive, a true negative, or a false negative. For the current study, a true positive was defined as

a participant who was employed at the time of neuropsychological assessment and for whom the

model-based prediction was also that the participant was employed. A false positive was defined

as a participant who was not employed at the time of neuropsychological assessment but for

whom the model-based prediction was that the participant was employed. A true negative was

defined as a participant who was not employed at the time of neuropsychological assessment and

for whom the model-based prediction was also that the participant was not employed. A false

negative was defined as a participant who was employed at the time of neuropsychological

assessment but for whom the model-based prediction was that the participant was not employed.

Following the identification of true positive, false positive, true negative, and false

negative predictions, the percentage of cases correctly classified, sensitivity, specificity, positive

predictive value, and negative predictive value of Model 1b for participants in Group 1 were

calculated. The percentage of cases correctly classified was defined as the percentage of model

predictions in Group 1 that correctly estimated the employment status of a participant at the time

of neuropsychological assessment. To calculate the percentage of cases correctly classified in

Group 1, the following formula was used:

true positives + true negatives / n

Reporting the accuracy of a model in terms of the percentage of cases correctly classified has

been a commonly used technique in previous return to work models (Drake et al., 2000; Fleming

et al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002; Simpson &

Schmitter-Edgecombe, 2002) and reporting a similar figure in the current study allowed for non-

statistical, subjective comparisons between models and with previous research.


Sensitivity was defined as the proportion of participants who were employed at the time

of neuropsychological assessment who were so identified by model predictions. To calculate the

sensitivity of Model 1b for participants in Group 1, the following formula was used:

true positives / (true positives + false negatives)

Specificity was defined as the proportion of participants who were not employed at the

time of neuropsychological assessment who were so identified by model predictions. To

calculate the specificity of Model 1b for participants in Group 1, the following formula was used:

true negatives / (true negatives + false positives)

Positive predictive value was defined as the proportion of participants classified as

employed by model predictions who were employed at the time of neuropsychological

assessment. To calculate the positive predictive value of Model 1b for participants in Group 1,

the following formula was used:

true positives / (true positives + false positives)

Negative predictive value was defined as the proportion of participants classified as not

employed by model predictions who were not employed at the time of neuropsychological

assessment. To calculate the negative predictive value of Model 1b for participants in Group 1,

the following formula was used:

true negatives / (true negatives + false negatives)

The same formulas were used throughout the study for the calculation of the percentage of cases

correctly classified, sensitivity, specificity, positive predictive value, and negative predictive

value.

Goal 3. Examine the degree to which the addition of the demographic variables of age,

premorbid occupation, education, number of subjective symptoms, ethnic status, and


independent driving status increases the proportion of variance accounted for, as well as the

percentage of cases correctly classified, the sensitivity, the specificity, the positive predictive

value, and the negative predictive value of the regression model described above.

In order to accomplish Goal 3, Model 2 was created using data from Group 1. Analysis

employed the use of hierarchical, forward-entry stepwise binary logistic regression. As

predictors of employment status at the time of neuropsychological assessment, Model 2 utilized

the same variables as Model 1b, with the addition of the demographic variables age, education in

years, number of subjective symptoms as measured by the Symptom Checklist-90-R Global

Severity Index, independent driving status, and premorbid occupation. Again, the use of Model

1b as the foundation for analysis was based on the robustness of that model when compared to

Model 1, as described above under Goal 2.

Predictor variables were entered into the regression analysis in two blocks, with

neuropsychological assessment variables entered in Block 1 and demographic variables entered

in Block 2. In each block, variables were entered using the same stepwise procedure described

for Model 1b (see Goal 1 above). The employment status of each participant at the time of

neuropsychological assessment was used as the criterion measure. The adjusted R2 value was

calculated using SPSS software during model creation as a measure of the amount of variance in

employment status that was accounted for by Model 2. Following the creation of Model 2, the

percentage of cases correctly classified, positive predictive value, negative predictive value,

sensitivity, and specificity in Group were 1 calculated.

While the ease of collecting demographic information reduces the clinical utility of the

analysis described in Goal 3, the analysis was conducted in order to identify which demographic

variables were most strongly related to returning to work following a TBI.


Goal 4. Examine the degree to which the addition of neuropsychological variables affects

the proportion of variance accounted for, percentage of cases correctly classified, sensitivity,

specificity, positive predictive value, and negative predictive value of a model which uses only

the demographic variables listed in goal three alone.

As in Goal 3, the ease of collecting demographic information reduces the clinical utility

of the analyses described in Goal 4. However, past research has suggested that demographic

variables alone may be sufficient to explain differences in employment status following TBI

(Arango-Lasprilla et al., 2009). These analyses were conducted in order to examine the degree to

which neuropsychological assessment variables account for incremental variability in

employment status following a TBI beyond that accounted for by demographic variables alone.

In order to accomplish Goal 4, Model 3 was created using hierarchical, forward-entry

stepwise binary logistic regression. As predictors of employment status at the time of

neuropsychological assessment, Model 3 utilized the same neuropsychological assessment

measures and demographic variable values from Group 1 that were used during the creation of

Model 2 (see Tables 5 and 6). Predictor variables were entered in two blocks, with demographic

variables being entered in Block 1 and neuropsychological assessment variables being entered in

Block 2. In each block, variables were entered using the same stepwise procedure described for

Model 1b (see Goal 1 above). The employment status of each participant at the time of

neuropsychological assessment was used as the criterion measure. The adjusted R2 value was

calculated using SPSS software during model creation as a measure of the amount of variance in

employment status that was accounted for by Model 3. Following the creation of Model 3, the

percentage of cases correctly classified, positive predictive value, negative predictive value,

sensitivity, and specificity of model predictions in Group 1 were calculated.


Goal 5. Calculate the proportion of variance accounted for, percentage of cases


value in order to examine the efficacy of the predictive models identified in goals one, three, and

four in an independent and age-matched sample taken from the same database.

The size of the database sample resulted in an undesirably small ratio of participants to

predictor variables, which can increase the probability of errors and decrease the accuracy of

population estimates. Confirmatory analyses were therefore employed as a measure of the

robustness and efficacy of Models 1b, 2, and 3. Following the creation of these models, three

separate employment group membership predictions were created for each participant in Group 2

(i.e. one prediction from each of the three models). Cross tabulations were then created to

compare the observed employment status of each participant in Group 2 to their employment

status as predicted by each model in order to identify true positive, false positive, true negative,

and false negative predictions.

The robustness of each model was measured by calculating the percentage of cases

correctly classified in Group 2. Next, the sensitivity, specificity, positive predictive value, and

negative predictive value of each model’s predictions in Group 2 were also calculated.

Data from participants in Group 2 were then entered into a forced-entry binary logistic

regression procedure. As a criterion variable, the procedure utilized the employment status of

each individual at the time of neuropsychological assessment. As predictors of employment

status at the time of neuropsychological assessment, the forced-entry procedure utilized the

neuropsychological assessment variables retained in Model 1b. This forced-entry procedure

identified the extent to which those same neuropsychological assessment variables accounted for

variance in employment status in Group 2. The same forced-entry procedure was repeated two


more times, once using the neuropsychological assessment and demographic variables retained

by Model 2, and once using the neuropsychological assessment and demographic variables

retained by Model 3.

In the case that the performance of the original prediction models was largely different

when confirmed in Group 2, a contingency plan was created to increase statistical power by

removing variables from the initial prediction models that showed low statistical significance and

repeating the confirmatory analysis.




four in two additional samples of litigants and non-litigants taken from the same database.

In order to accomplish Goal 6, additional confirmatory analyses were performed in order

to examine the extent to which Models 1b, 2, and 3 would be confirmed in groups 3 and 4. The

efficacy of Models 1b, 2 and 3 were tested in Groups 3 and 4 by creating cross tabulations that

compared actual employment status to employment status as predicted by each model. These

cross tabulations were used to calculate the number of true positive, false positive, true negative,

and false negative predictions from each model in groups 3 and 4. The percentage of cases

correctly classified in Groups 3 and 4 was then calculated, along with the sensitivity, specificity,

positive predictive value, and negative predictive value of each model’s predictions in the same

groups.

Data from participants in Group 3 were then entered into a forced-entry binary logistic

regression procedure. As a criterion variable, the procedure utilized the employment status of

each individual at the time of neuropsychological assessment. As predictors of employment


status at the time of neuropsychological assessment, the forced-entry procedure utilized the

neuropsychological assessment variables retained in Model 1b. This forced-entry procedure

identified the extent to which those same neuropsychological assessment variables accounted for

variance in employment status in Group 3. The same forced-entry procedure was repeated two

more times, once using the neuropsychological assessment and demographic variables retained

by Model 2, and once using the neuropsychological assessment and demographic variables

retained by Model 3. This same forced-entry procedure was then repeated to mimic the creation

of Models 1b, 2, and 3 using data from Group 4.

Goal 7. Examine any changes in the proportion of variance in employment status

accounted for, correct classification rate, sensitivity, specificity, positive predictive value,

negative predictive value, or in the predictor variables that account for a significant amount of

variance in employment status when the models mentioned in goals three and four are created in

a sample of 76 litigants taken from the study database.

In order to accomplish Goal 7, two additional models (Model 4 and Model 5) were

created using data from the 76 litigants comprising Group 3. The same exploratory hierarchical

stepwise binary logistic regression methods described for the creation of Models 2 and 3 were

used for the creation of Models 4 and 5, respectively (see Goal 3 and Goal 4 above). Two

separate employment status predictions were made for each participant in Group 3 using Models

4 and 5, and cross tabulations were created in order to compare the observed employment status

of each participant in Group 3 at the time of neuropsychological assessment with their

employment status as predicted by Models 4 and 5. These cross tabulations allowed for the

calculation of true positive, false positive, true negative, and false negative predictions.

Following these calculations, the percentage of cases in Group 3 correctly classified by Models 4


and 5 was determined. The sensitivity, specificity, positive predictive value, and negative

predictive value of each model’s predictions in Group 3 were also calculated. Subjective

comparisons of Models 4 and 5 with Models 2 and 3 were conducted by observing the calculated

correct classification rate, sensitivity, specificity, positive predictive power, and negative

predictive power for each model and determining which models produced higher relative values.

Observational comparisons were also made between Models 2, 3, 4, and 5 by comparing the

demographic and neuropsychological assessment variables that were retained by each model.

This was done in order to provide insight into the effects that litigation status may have on the

neuropsychological and demographic variables that are related to employment status following

TBI.

The adjusted R2 values of Models 4 and 5 were calculated using SPSS software during

model creation as a measure of the amount of variance in employment status that was accounted

for by each model.




variance in employment status when the models mentioned in goals three and four are created

using data from all cases contained in the study sample.

In order to accomplish Goal 8 and to increase sample size and thus maximize statistical

power, two additional models were created (Model 6 and Model 7). The creation of Models 6

and 7 mimicked the procedures used for Models 2 and 3, respectively (see Goal 3 and Goal 4

above), but was created using data from the entire sample of 192 participants. Two separate

employment status predictions were made for each participant in the study sample using Models


6 and 7, and cross tabulations were created to compare the observed employment status of each

participant in the study sample with their employment status as predicted by Models 6 and 7.

These cross tabulations allowed for the calculation of true positive, false positive, true negative,

and false negative predictions. Following these calculations, the percentage of cases in the study

sample correctly classified by Models 6 and 7 was determined. The sensitivity, specificity,

positive predictive value, and negative predictive value of each model’s predictions in the full

study sample were also calculated. Nonstatistical comparisons of Model 6 and 7 with Models 1b

through 5 were then conducted to provide insight into the effects of increased sample size and

statistical power on the neuropsychological and demographic variables that are related to

employment status following TBI. These comparisons were made using the same methods as

described above under Goal 7. The adjusted R2 values of Models 6 and 7 were calculated using

SPSS software during model creation as a measure of the amount of variance in employment

status that was accounted for by each model.

Following the creation of each model, power analysis was conducted using Statistics

Calculator, Version 3.0 (Soper, 2012) to ensure that the size of the database sample was

sufficient to detect significant contributions from predictor variables.

Results

Basic Data

A summary of the neuropsychological test scores for participants in Groups 1 through 4

can be found below in Table 7. Results of MANOVA analysis were non-significant (λ = 0.90, p

= 0.97) indicating that differences between Groups 1, 2, 3, and 4 on the neuropsychological

variables of interest did not exceed those expected by chance.


Table 7

Neuropsychological Variables of Interest in Groups 1-4 and the Full Study Sample

Group 1 Group 2 Group 3 Group 4

Variable Mean SD Mean SD Mean SD Mean SD

Months since injury 11.0 14.5 9.1 11.9 10.7 11.6 9.5 14.3

RAVLT long-delay free recall* 32.6 15.0 30.5 15.1 34.2 14.1 29.6 15.4

RAVLT recognition* 43.0 11.7 39.4 14.1 42.5 13.0 40.4 13.1

RCFT copy* 33.7 19.2 33.0 19.0 34.8 16.9 32.3 20.4

RCFT immediate recall* 38.8 18.1 36.1 19.7 41.2 16.5 34.9 20.1

RCFT delayed recall* 37.8 17.8 35.3 19.3 40.2 17.0 34.1 19.2

RCFT recognition total correct* 39.6 17.4 40.3 16.6 42.1 15.5 38.5 17.8

WAIS-III Digit-Symbol Coding* 39.9 8.7 39.4 9.6 40.5 7.8 39.0 9.9

Trail Making, Part B* 43.0 11.0 39.2 14.9 43.0 11.3 39.8 14.3

Judgment of Line Orientation* 50.3 9.4 49.1 9.5 49.7 8.3 49.6 10.1

WAIS-III Block Design* 47.1 10.1 45.3 9.7 46.7 8.4 45.8 10.8

Category Test* 38.9 11.6 38.9 13.0 37.7 11.6 39.5 12.8

WAIS-III Similarities* 45.1 8.3 44.8 7.8 45.4 8.6 44.6 7.6

WAIS-III FSIQ 90.2 13.2 89.0 12.8 90.1 11.6 89.0 13.6

WAIS-III PIQ 90.9 16.4 88.7 15.9 90.3 14.5 89.2 17.0

WAIS-III VIQ 91.0 11.3 90.6 12.1 91.3 11.1 90.3 12.0

OTBM 41.9 6.5 40.0 7.5 42.0 5.4 40.2 7.9


Table 7 (Cont.)

Group 1 Group 2 Group 3 Group 4

Variable Mean SD Mean SD Mean SD Mean SD

COWAT* 37.6 8.6 37.1 9.0 38.6 8.1 36.4 9.1

Finger Tapping: dominant* 40.0 9.9 37.8 11.7 39.6 9.2 38.3 11.8

Finger Tapping: non-dominant* 42.2 11.2 40.4 13.2 41.8 11.5 40.9 12.8

Note: An asterisk (*) indicates a T-score value. Results of MANOVA showed all

between-group differences on the variables of interest to be non-significant. RAVLT =

Rey Auditory Verbal Learning Test; RCFT = Rey Complex Figure Test; WAIS-III =

Wechsler Adult Intelligence Test, Third Edition; FSIQ = Full-scale Intelligence

Quotient; PIQ = Performance Intelligence Quotient; VIQ = Verbal Intelligence Quotient;

COWAT = Controlled Oral Word Association Test.

Initial Analyses

As stated above, statistically significant differences between Groups 1 through 4 on the

demographic, neuropsychological assessment, and employment variables used in analysis were

assessed using MANOVA and found to be non-significant (see Tables 5 and 7). Initial Box-

Tidwell analysis showed no significant contributions to a regression model from the interaction

terms of each predictor and its natural logarithm, indicating that there was no violation of

linearity in the logit (Tabachnick & Fidell, 2007). Results are organized below according to the

eight study goals outlined above under Goals.


Study Goals

Goal 1. Employ exploratory step-wise regression procedures in order to examine the

incremental proportion of variance in employment status accounted for with the step-wise

addition of each neuropsychological assessment variable, and identify the most parsimonious

model (i.e., the model that includes variables beyond which inclusion of additional variables


In order to accomplish Goal 1, Model 1 was created using data from the 96 participants

comprising Group 1. Significant χ2 goodness-of-fit tests (p < 0.01) justified the addition of

neuropsychological assessment variables to a constant-only model. The regression model was

constructed using a forward selection step-wise procedure with χ2 selection criteria. During Step

1, the model retained the OTBM as a significant predictor of employment status, resulting in a

Cox and Snell R2 value of 0.21 and a χ2 change score of 22.92 (p < 0.01). In Step 2, the Judgment

of Line Orientation T-score was retained in addition to the OTBM, resulting in a Cox and Snell

R2 value of 0.23 (R2 change = 0.02) and a χ2 change score of 2.49 (p = 0.12). The resulting non-

significant χ2 change score during Step 2 indicated that the inclusion of additional variables did

not account for a significant portion of additional variance in employment status, and model

creation was halted following Step 1.

Hosmer and Lemeshow tests indicated a good fit for this one-variable model (p = 0.51),

and power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed an

obtained statistical power of 0.99. A summary of the two steps used to create Model 1, including

the χ2 change for each step, the significance of the χ2 change for each step, Cox and Snell R2

values for each step, the Cox and Snell R2 change value for each step, and the odds ratios for

each variable can be found in Table 8.


Table 8

Summary of the Two Steps Completed During Creation of Model 1

Variable χ2change p R2 R2

change β Exp(β)

Step 1 22.92 <0.01 0.21

OTBM 0.19 1.21

Step 2 2.49 0.12 0.23 0.02

OTBM 0.14 1.15

JOLO 0.05 1.06

Note: p values listed in the table represent the statistical significance of the χ2 change

for each step of model creation. JOLO = Judgment of Line Orientation Test.

As mentioned above in Data Analysis, Model 1b was created in order to examine the

aggregated effect of including additional neuropsychological predictor variables to Model 1.

Model 1b was created by repeating the analyses described above and allowing variable inclusion

to continue until the specified selection criteria did not result in the inclusion of any additional

variables. Model creation was allowed to continue regardless of the significance of the χ2 change

statistic for each step. The resulting model for predicting employment status retained three

variables in addition to the constant, the Ward 7-subtest PIQ, the Judgment of Line Orientation

T-score, and the Finger Tapping Test T-score from the dominant hand. The OTBM, which had

been included in Step 1 as described above, was removed during the final step of model creation

due to low statistical significance following the inclusion of additional variables. While χ2

change scores had not reached statistical significance during Step 2 (p = 0.12), Step 3 (p = 0.07),

or Step 4 (p = 0.10), post-hoc analysis showed that the χ2 change score between step 1 and step 4

was significant (p < 0.05), suggesting that the cumulative effect of including three additional

variables had accounted for a significant change in the amount of variance accounted for in


employment status. In Step 5, the removal of the OTBM resulted in a non-significant χ2 change

score (p = 0.98).This finding indicated that the amount of variance in employment status

accounted for was not significantly changed, and thus justified the removal of the variable.

Hosmer and Lemeshow tests indicated a good fit for this three-variable model (p > 0.05),

and power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed an

obtained statistical power of 0.95. A summary of the five steps used to create Model 1b,

including the χ2 change for each step, the significance of the χ2 change for each step, Cox and

Snell R2 values for each step, the Cox and Snell R2 change value for each step, and the odds

ratios for each variable can be found in Table 9.


Table 9

Summary of the Five Steps Completed During Creation of Model 1b


change β Exp(β)

Step 1 22.92 <0.01 0.21

OTBM 0.19 1.21

Step 2 2.49 0.12 0.23 0.02

OTBM 0.14 1.15

JOLO 0.05 1.06

Step 3 3.28 0.07 0.26 0.03

OTBM 0.10 1.10

JOLO 0.07 1.07

Tapping-Dominant 0.06 1.06

Step 4 2.65 0.10 0.28 0.02

OTBM <0.01 1.00

JOLO 0.07 1.07


PIQ 0.05 1.05

Step 5 <0.01 0.98 0.28 0.00

JOLO 0.07 1.07


PIQ 0.05 1.05

Note: p values listed in the table represent the statistical significance of the χ2 change for

each step of model creation. JOLO = Judgment of Line Orientation Test; Tapping-

Dominant = Finger Tapping Test, dominant hand score; PIQ = Performance Intelligence

Quotient.


Goal 2. Calculate the percentage of cases correctly classified, sensitivity, specificity,

positive predictive value, and negative predictive value in order to examine the extent to which

the regression model mentioned above is able to correctly estimate an individual’s employment

status at the time of assessment.

Model 1b consisted of the Ward 7-subtest PIQ score, the Judgment of Line Orientation T-

score, and the Finger Tapping Test T-score from the dominant hand. Using the analytic formulas

presented under Data Analysis, Goal 2 above, Model 1b showed a correct classification rate of

83.3% for employment status in Group 1, with 93.8% sensitivity and 61.3% specificity. In

addition, Model 1b demonstrated a positive predictive value of 83.6% and a negative predictive

value of 82.6% within Group 1.

Goal 3. Examine the degree to which the addition of the demographic variables of age,

premorbid occupation, education, number of subjective symptoms, ethnic status, and

independent driving status increases the proportion of variance accounted for, as well as the

percentage of cases correctly classified, the sensitivity, the specificity, the positive predictive

value, and the negative predictive value of the regression model described above.


comprising Group 1. Initial classification tables showed that the creation of a constant-only

model that categorized all participants as employed at the time of their assessment resulted in a

67.7% correct classification rate in Group 1. Chi-squared goodness-of-fit tests were significant (p

< 0.01), thus justifying the addition of neuropsychological assessment variables to the constant-

only model.


Model 2 was created in two blocks. The first block was completed in five steps that

mimicked the creation of Model 1b (see Goal 1 above). At the end of the first block, Model 2

was identical to Model 1b.

Following the first block, significant χ2 goodness-of-fit tests (p = 0.01) justified the

inclusion of additional, demographic predictor variables. The second block was completed in two

steps. In the first step, premorbid occupation was retained as a predictor variable, resulting in an

84.4% correct classification rate in Group 1 and a Cox and Snell R2 value of 0.35. In the second

step, independent driving status was retained as an additional predictor variable, resulting in an

86.5% correct classification rate and a Cox and Snell R2 value of 0.36 in Group 1.

The final model consisted of the WAIS-III PIQ (p = 0.03), the Judgment of Line

Orientation T-score (p = 0.12), the Finger Tapping Test dominant hand T-score (p = 0.03),

premorbid occupation (semi-skilled p < 0.01; skilled p = 0.01), and independent driving status

(driving p = 0.09). Hosmer and Lemeshow tests indicated a good fit for the final model (p =

0.09). Using the analytic formulas described above under Data Analysis, Goal 2, Model 2

showed a correct classification rate of 86.5% in Group 1, with 95.4% sensitivity and 67.7%

specificity. In addition, Model 2 demonstrated a positive predictive value of 86.1% and a

negative predictive value of 87.5% within Group 1.

Power analysis conducted using Statistics Calculators, Version 3.0 (Soper, 2012) showed

an obtained statistical power of 0.99 for Model 2. A summary of the seven steps used to create

Model 2, including χ2 change scores, significance of χ2 change scores, Cox and Snell R2 values,

Cox and Snell R2 change scores for each step, and the odds ratios for each variable can be found

in Table 10.


Table 10

Summary of the Seven Steps Completed During Creation of Model 2


change β Exp(β)

Step 1 22.92 <0.01 0.21

OTBM 0.19 1.21

Step 2 2.49 0.12 0.23 0.02

OTBM 0.14 1.15

JOLO 0.05 1.06

Step 3 3.28 0.07 0.26 0.03

OTBM 0.10 1.10

JOLO 0.07 1.07


Step 4 2.65 0.10 0.28 0.02

OTBM <0.01 1.00

JOLO 0.07 1.07


PIQ 0.05 1.05

Step 5 <0.01 0.98 0.28 0.00

JOLO 0.07 1.07


PIQ 0.05 1.05

Step 6 9.24 0.01 0.35 0.07

JOLO 0.06 1.06


PIQ 0.06 1.06

Premorbid Employment

Semi-skilled 2.42 11.29

Skilled 2.40 11.01


Table 10 (Cont.)


change β Exp(β)

Step 7 2.92 0.09 0.36 0.01

JOLO 0.06 1.06


PIQ 0.05 1.06



Skilled 2.49 12.01

Independent Driving 1.46 4.29

Note: p values presented represent the statistical significance of the χ2 change for each

step of model creation. JOLO = Judgment of Line Orientation Test; Tapping-dominant =

Finger Tapping Test, dominant hand; PIQ = Performance Intelligence Quotient.

Goal 4. Examine the degree to which the addition of neuropsychological variables affects the

proportion of variance accounted for, percentage of cases correctly classified, sensitivity,

specificity, positive predictive value, and negative predictive value of a model which uses only

the demographic variables listed in goal three alone.


comprising Group 1. Initial classification tables showed that the creation of a constant-only

model that categorized all participants as employed at the time of their assessment resulted in a

67.7% correct classification rate in Group 1. Chi-squared goodness-of-fit tests were significant (p

< 0.01), thus justifying the addition of demographic variables to the constant-only model.

Model 3 was created in two blocks. The first block was completed in three steps. In the

first step, analysis retained independent driving status as a predictor of employment status,

resulting in a 74.0% correct classification rate in Group 1 and a Cox and Snell R2 value of 0.11.

During the second step, analysis retained premorbid occupation as an additional predictor of


employment status, resulting in a 76.0% correct classification rate in Group 1 and a Cox and

Snell R2 value of 0.19. In the third step, education was retained as an additional predictor

variable, resulting in a 75.0% correct classification rate in Group 1 and a Cox and Snell R2 value

of 0.23.

Following the first block, significant (p < 0.01) χ2 goodness-of-fit tests justified the

inclusion of neuropsychological assessment predictor variables. The second block was

completed in four steps. In the first step, the WAIS-III PIQ was retained as a predictor of

employment status, resulting in an 82.3% correct classification rate in Group 1 and a Cox and

Snell R2 value of 0.32. During the second step, the Finger Tapping Test dominant hand T-score

was retained as a predictor variable, resulting in an 86.5% correct classification rate in Group 1

and a Cox and Snell R2 value of 0.36. In the third step, the Rey Complex Figure Test recognition

trial T-score was retained as an additional predictor of employment status, resulting in an 88.5%

correct classification rate in Group 1 and a Cox and Snell R2 value of 0.39. In the fourth step, the

COWAT T-score was retained as a final predictor variable, resulting in an 88.5% correct-

classification rate in Group 1 and a Cox and Snell R2 value of 0.40.

As predictor variables, the final model included education (p = 0.11), premorbid

occupation (semi-skilled p < 0.01; skilled p < 0.01), independent driving status (driving p =

0.04), PIQ (p = 0.01), COWAT T-score (p = 0.12), Finger Tapping Test dominant hand T-score

(p = 0.02), and the Rey Complex Figure recognition trial T-score (p = 0.05). Hosmer and

Lemeshow tests showed a poor fit for the final model (p < 0.01), indicating the possibility of

uneven distribution of model predictions across deciles of risk. Using the analytic formulas

presented above under Data Analysis, Goal 2, Model 3 showed a correct classification rate of

88.5% in Group 1, with 98.5% sensitivity and 67.7% specificity. In addition, Model 3


demonstrated a positive predictive value of 86.5% and a negative predictive value of 95.5%

within Group 1.


an obtained statistical power of 0.99 for Model 3. A summary of the seven steps used to create


Cox and Snell R2 change scores for each step, and the odds ratios for each variabe can be found

in Table 11.


Table 11

Summary of the Seven Steps Completed During Creation of Model 3


change β Exp(β)

Step 1 10.72 <0.01 0.11


Step 2 9.76 0.01 0.19 0.08




Skilled 2.34 10.35

Step 3 4.85 0.03 0.23 0.04




Skilled 2.54 12.64

Education 0.39 1.48

Step 4 12.33 <0.01 0.32 0.09




Skilled 2.65 14.18

Education 0.31 1.36

PIQ 0.07 1.07

Step 5 5.78 0.02 0.36 0.04




Skilled 2.72 15.25

Education 0.33 1.39

PIQ 0.06 1.06



Table 11 (Cont.)


change β Exp(β)

Step 6 3.16 0.08 0.39 0.03




Skilled 2.88 17.84

Education 0.31 1.36

PIQ 0.06 1.06


RCFT-Recognition 0.04 1.04

Step 7 2.69 0.10 0.40 0.01




Skilled 3.30 27.05

Education 0.36 1.44

PIQ 0.07 1.07



COWAT -0.08 0.92


step of model creation. PIQ = Performance Intelligence Quotient; Tapping – dominant =

Finger Tapping Test, dominant hand; RCFT = Rey Complex Figure Test; COWAT =

Controlled Oral Word Association Test.





four in an independent and age-matched sample taken from the same database.

Initial calculations showed that classifying all members of Group 2 as employed would

result in a 58.3% correct classification rate. Using the analytical formulas presented above under

Data Analysis, Goal 2, Model 1b showed a 69.8% correct classification rate overall, with 85.7%

sensitivity and 47.5% specificity in Group 2. Model 1b had a positive predictive value of 69.6%

and a negative predictive value of 70.4% in Group 2.

Model 2 showed a 72.9% correct classification rate overall in Group 2, with 87.5%

sensitivity and 52.5% specificity. Model 2 had a positive predictive value of 72.1% and a

negative predictive value of 75.0% in Group 2.

Model 3 showed a 71.9% correct classification rate overall in Group 2, with 83.9%

sensitivity and 55.0% specificity. Model 3 had a positive predictive value of 72.3% and a

negative predictive value of 71.0% in Group 2.

Cox and Snell R2 values for Models 1b, 2, and 3 in Group 2 were 0.18, 0.28, and 0.29,

respectively. Table 12 shows the correct classification rates, sensitivity, specificity, positive

predictive value, negative predictive value, and Cox and Snell R2 value for Models 1b, 2, and 3

in Groups 1 and 2. The performance of the initial prediction models in Group 2 was considered

subjectively similar enough to their performance in Group 1 that further modification of the

models was deemed unnecessary.


Table 12

Statistical Performance of Models 1b, 2, and 3 in

Groups 1 and 2

Group 1 Group 2

Correct Classification Rate

Model 1b 83.3 69.8

Model 2 86.5 72.9

Model 3 88.5 71.9

Sensitivity

Model 1b 93.8 85.7

Model 2 95.4 87.5

Model 3 98.5 83.9

Specificity

Model 1b 61.3 47.5

Model 2 67.7 52.5

Model 3 67.7 55.0

Positive Predictive Value

Model 1b 83.6 69.6

Model 2 86.1 72.1

Model 3 86.5 72.3

Negative Predictive Value

Model 1b 82.6 70.4

Model 2 87.5 75.0

Model 3 95.5 71.0

R2

Model 1b 0.28 0.18

Model 2 0.36 0.28

Model 3 0.40 0.29

Note: Group 1 includes the 96 cases from which Models

1b, 2, and 3 were created, while Group 2 includes the 96

cases that were not included in the creation of Models

1b, 2, and 3.


Goal 6. Calculate the proportion of variance accounted for, percentage of cases correctly

classified, sensitivity, specificity, positive predictive value, and negative predictive value in order

to examine the efficacy of the predictive models identified in goals one, three, and four in two

additional samples of litigants and non-litigants taken from the same database.

Initial calculations showed that classifying all members of Group 3 as employed would

result in a 59.2% correct classification rate, while classifying all members of Group 4 as

employed would result in a 65.2% correct classification rate.

In Group 3, Model 1b showed a 76.3% correct classification rate with 93.3% sensitivity

and 51.6% specificity. Model 1b demonstrated a positive predictive value of 73.7%, and a

negative predictive value of 84.2% in Group 3. In Group 4, Model 1b showed a correct

classification rate of 79.1%, with 88.0% sensitivity and 62.5% specificity. Model 1b also

demonstrated a positive predictive value of 81.5%, and a negative predictive value of 73.5% in

Group 4.

In Group 3, Model 2 showed a 76.3% correct classification rate with 93.3% sensitivity

and 51.6% specificity. Model 2 demonstrated a positive predictive value of 73.7% and a negative

predictive value of 84.2% in Group 3. In Group 4, Model 2 showed a correct classification rate

of 79.1%, with 88.0% sensitivity and 62.5% specificity. Model 2 also demonstrated a positive

predictive value of 81.5% and a negative predictive value of 73.5% in Group 4.

In Group 3, Model 3 showed a 76.3% correct classification rate with 97.8% sensitivity

and 45.2% specificity. Model 3 demonstrated a positive predictive value of 72.1% and a negative

predictive value of 93.3% in Group 3. In Group 4, Model 3 showed a correct classification rate

of 82.6%, with 88.0% sensitivity and 72.5% specificity. Model 3 also demonstrated a positive

predictive value of 85.7% and a negative predictive value of 76.3% in Group 4.


In Group 3, Models 1b, 2, and 3 showed Cox and Snell R2 values of 0.18, 0.28, and 0.35,

respectively. In Group 4, the same models showed Cox and Snell R2 values of 0.22, 0.31, and

0.33, respectively. Table 13 outlines the performance of Models 1, 2, and 3 in Groups 3 and 4.


Table 13

Statistical Performance of Models 1b through 3 in

Groups 3 and 4

Group 3 Group 4

Correct Classification Rate

Model 1b 76.3 79.1

Model 2 76.3 79.1

Model 3 76.3 82.6

Sensitivity

Model 1b 93.3 88.0

Model 2 93.3 88.0

Model 3 97.8 88.0

Specificity

Model 1b 51.6 62.5

Model 2 51.6 62.5

Model 3 45.2 72.5

Positive Predictive Value

Model 1b 73.7 81.5

Model 2 73.7 81.5

Model 3 72.1 85.7

Negative Predictive Value

Model 1b 84.2 73.5

Model 2 84.2 73.5

Model 3 93.3 76.3

R2

Model 1b 0.18 0.22

Model 2 0.28 0.31

Model 3 0.35 0.33

Note: Group 3 includes the 76 participants from the study

sample who were involved in litigation at the time of

assessment, while Group 4 includes the 115 participants

from the study sample that were not involved in litigation

at the time of assessment.





variance in employment status when the models mentioned in goals three and four are created in

a sample of 76 litigants taken from the study database.

Model 4. Initial classification tables showed that the creation of a constant-only model

that categorized all participants as employed at the time of their assessment resulted in a 59.2%

correct classification rate for Group 3. Chi-squared goodness-of-fit tests were significant (p <

0.01), thus justifying the addition of neuropsychological assessment variables to the constant-

only model.

Model 4 was created in two blocks. The first block was completed in four steps. In the

first step, the OTBM was retained in addition to the constant as a predictor variable, resulting in

a 72.4% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.20. In the second

step, number of months since injury was retained as a predictor variable, resulting in a 71.1%

correct classification rate in Group 3 and a Cox and Snell R2 value of 0.24. The Rey Complex

Figure Test delayed recall trial T-score was retained as a predictor variable during the third step,


Finally, during the fourth step, the Rey Complex Figure Test recognition trial T-score was

retained as a predictor variable, resulting in a 78.9% correct classification rate and a Cox and

Snell R2 value of 0.30.

Following the first block, significant χ2 goodness-of-fit tests (p = 0.01) justified the


steps. In the first step, education was retained as a predictor variable, resulting in an 81.6%


correct classification rate in Group 3 and a Cox and Snell R2 value of 0.35. In the second step,

age was retained as an additional predictor variable, resulting in an 81.6% correct classification

rate in Group 3 and a Cox and Snell R2 value of 0.39.

The final model consisted of number of months since injury (p = 0.04), the Rey Complex

Figure Test delayed recall trial T-score (p = 0.17), the Rey Complex Figure Test recognition trial

T-score (p = 0.12), the OTBM (p < 0.01), age (p = 0.04), and education (p = 0.02). Hosmer and

Lemeshow tests indicated a good fit for the final model (p = 0.79). Using the analytical formulas

presented above under Data Analysis, Goal 2, Model 4 showed a correct classification rate of

81.6% in Group 3, with 84.4% sensitivity and 77.4% specificity. In addition, Model 4

demonstrated a positive predictive value of 84.4% and a negative predictive value of 77.4%

within Group 3. Model 4 demonstrated a Cox and Snell R2 value of 0.39.


an obtained statistical power of 0.97 for Model 4. A summary of the six steps used to create



in Table 14.


Table 14

Summary of the Six Steps Completed During Creation of Model 4


change β Exp(β)

Step 1 17.01 <0.01 0.20

OTBM 0.21 1.23

Step 2 4.17 0.04 0.24 0.04

OTBM 0.19 1.21

Months Since Injury 0.05 1.05

Step 3 2.94 0.09 0.27 0.03

OTBM 0.30 1.35


RCFT-Delayed -0.04 0.96

Step 4 2.82 0.09 0.30 0.03

OTBM 0.27 1.31




Step 5 6.14 0.01 0.35 0.05

OTBM 0.29 1.33




Education 0.45 1.56

Step 6 4.55 0.03 0.39 0.04

OTBM 0.28 1.33




Education 0.52 1.68

Age -0.06 0.94


step of model creation. RCFT-Delayed = Rey Complex Figure Test delayed recall trial;

RCFT-Recognition = Rey Complex Figure Test recognition trial.


Model 5. Initial classification tables showed that the creation of a constant-only model

that categorized all participants as employed at the time of their assessment resulted in a 59.2%

correct classification rate for Group 3. Chi-squared goodness-of-fit tests were significant (p <

0.01), thus justifying the addition of demographic predictor variables to the constant-only model.

Model 5 was created in two blocks. The first block was completed in one step, which

retained education as a predictor variable in addition to the constant. The first block resulted in a

64.5% correct classification rate in Group 3 and a Cox and Snell R2 value of 0.13.


inclusion of additional, neuropsychological assessment predictor variables. The second block

was completed in four steps. In the first step, the OTBM was retained as a predictor variable,


In the second step, the Rey Complex Figure Test delayed recall trial T-score was retained as an

additional predictor variable, resulting in a 73.7% correct classification rate in Group 3 and a

Cox and Snell R2 value of 0.30. During the third step, the Category Test T-score was retained as

a predictor variable, resulting in a 76.3% correct classification rate and a Cox and Snell R2 value

of 0.34. During the fourth step, number of months since injury was retained as an additional

predictor variable, resulting in an 80.3% correct classification rate and a Cox and Snell R2 value

of 0.36.

The final model consisted of number of months since injury (p = 0.13), the Rey Complex

Figure Test delayed recall trial T-score (p = 0.03), the Category Test T-score (p = 0.10), the

OTBM (p < 0.01), and education (p = 0.01). Hosmer and Lemeshow tests indicated a good fit for

the final model (p = 0.69), indicating an even distribution of model predictions across deciles of

risk. Model 5 showed a correct classification rate of 80.3% in Group 3, with 88.9% sensitivity


and 67.7% specificity. In addition, Model 5 demonstrated a positive predictive value of 80.0%

and a negative predictive value of 80.8% within Group 3. Model 5 demonstrated a Cox and Snell

R2 value of 0.36.


an obtained statistical power of 0.94 for Model 5. A summary of the five steps used to create



in Table 15.


Table 15

Summary of the Five Steps Completed During Creation of Model 5


change β Exp(β)

Step 1 10.44 <0.01 0.13

Education 0.48 1.62

Step 2 13.87 <0.01 0.27 0.14

Education 0.46 1.59

OTBM 0.20 1.22

Step 3 3.03 0.08 0.30 0.03

Education 0.51 1.67

OTBM 0.30 1.35


Step 4 3.83 0.05 0.34 0.04

Education 0.63 1.88

OTBM 0.28 1.32


Category Test 0.06 1.06

Step 5 2.63 0.11 0.36 0.02

Education 0.56 1.75

OTBM 0.28 1.33


Category Test 0.05 1.05



step of model creation. RCFT-Delayed = Rey Complex Figure Test delayed recall trial.





variance in employment status when the models mentioned in goals three and four are created

using data from all cases contained in the study sample.

Model 6. In order to accomplish Goal 8, Model 6 was created using data from the entire

sample of 192 participants. Initial classification tables showed that the creation of a constant-

only model that categorized all participants as employed at the time of their assessment resulted

in a 63.0% correct classification rate for the study sample. Chi-squared goodness-of-fit tests were

significant (p < 0.01), thus justifying the addition of neuropsychological assessment variables to

the constant-only model.

Model 6 was created in two blocks. The first block was completed in three steps. In the

first step, the OTBM was retained in addition to the constant as a predictor variable, resulting in

a 76.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.21. In

the second step, the WAIS-III PIQ was retained as a predictor variable, resulting in a 78.1%

correct classification rate in the study sample and a Cox and Snell R2 value of 0.22. The

COWAT T-score was retained as a predictor variable during the third step, resulting in a 78.1%

correct classification rate in the study sample and a Cox and Snell R2 value of 0.23.



steps. In the first step, premorbid occupation was retained as a predictor variable, resulting in a

77.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.30. In the


second step, independent driving status was retained as an additional predictor variable, resulting

in a 78.6% correct classification rate in the study sample and a Cox and Snell R2 value of 0.32.

The final model consisted of the WAIS-III PIQ (p = 0.15), the COWAT T-score (p =

0.07), the OTBM (p = 0.02), premorbid occupation (semi-skilled p < 0.01; skilled p < 0.01), and

independent driving status (driving p = 0.06). Hosmer and Lemeshow tests indicated a good fit

for the final model (p = 0.26). Model 6 showed a correct classification rate of 78.6% within the

full sample, with 91.7% sensitivity and 56.3% specificity. In addition, Model 6 demonstrated a

positive predictive value of 78.2% and a negative predictive value of 80.0% within the full study

sample. Model 6 demonstrated a Cox and Snell R2 value of 0.32.


an obtained statistical power of 0.99 for Model 6. A summary of the five steps used to create



in Table 16.


Table 16

Summary of the Five Steps Completed During Creation of Model 6


change β Exp(β)

Step 1 45.42 <0.01 0.21

OTBM 0.16 1.18

Step 2 3.20 0.07 0.22 0.01

OTBM 0.11 1.12

PIQ 0.03 1.03

Step 3 2.21 0.13 0.23 0.01

OTBM 0.15 1.16

PIQ 0.03 1.03

COWAT -0.04 0.96

Step 4 18.23 <0.01 0.30 0.07

OTBM 0.15 1.16

PIQ 0.03 1.03

COWAT -0.04 0.96



Skilled 2.45 11.58

Step 5 3.65 0.06 0.32 0.02

OTBM 0.13 1.14

PIQ 0.03 1.03

COWAT -0.05 0.95



Skilled 2.51 12.31



step of model creation. PIQ = Performance Intelligence Quotient; COWAT = Controlled

Oral Word Association Test.


Model 7. In order to accomplish Goal 8, Model 7 was created using data from the entire

study sample of 192 participants. Initial classification tables showed that the creation of a

constant-only model that categorized all participants as employed at the time of their assessment

resulted in a 63.0% correct classification rate for the study sample. Chi-squared goodness-of-fit

tests were significant (p < 0.01), thus justifying the addition of demographic predictor variables

to the constant-only model.

Model 7 was created in two blocks. The first block was completed in three steps. The first

step retained premorbid occupation as a predictor variable in addition to the constant, resulting

in a 72.4% correct classification rate in the study sample and a Cox and Snell R2 value of 0.15.

During the second step, independent driving status was retained as an additional predictor

variable, resulting in a 75.5% correct classification rate in the study sample and a Cox and Snell

R2 value of 0.22. In the third step, education was retained as a predictor variable, resulting in a

75.5% correct classification rate in the study sample and a Cox and Snell R2 value of 0.23.


inclusion of additional, neuropsychological assessment predictor variables. The second block

was completed in five steps. In the first step, the OTBM was retained as a predictor variable,

resulting in a 78.6% correct classification rate in the study sample and a Cox and Snell R2 value

of 0.30. In the second step, the COWAT T-score was retained as an additional predictor variable,

resulting in a 77.6% correct classification rate in the study sample and a Cox and Snell R2 value

of 0.31. During the third step, the Rey Complex Figure Test immediate recall trial T-score was

retained as a predictor variable, resulting in a 79.7% correct classification rate and a Cox and

Snell R2 value of 0.32. During the fourth step, the Rey Complex Figure Test recognition trial T-

score was retained as an additional predictor variable, resulting in a 78.6% correct classification


rate in the study sample and a Cox and Snell R2 value of 0.34. Finally, number of months since

injury was retained as a predictor variable, resulting in a 79.2% correct classification rate in the

study sample and a Cox and Snell R2 value of 0.35.

The final model consisted of education (p = 0.52), premorbid occupation (semi skilled p

< 0.01; skilled p < 0.01), independent driving status (driving p = 0.09), number of months since

injury (p = 0.09), the COWAT T-score (p = 0.04), the Rey Complex Figure Test immediate

recall trial T-score (p = 0.05), the Rey Complex Figure Test recognition trial T-score (p = 0.06),

and the OTBM (p < 0.01). Hosmer and Lemeshow tests indicated a good fit for the final model

(p = 0.35), indicating an even distribution of model predictions across deciles of risk. Model 7

showed a correct classification rate of 79.2% within the full sample, with 89.3% sensitivity and

62.0% specificity. In addition, Model 7 demonstrated a positive predictive value of 80.0% and a

negative predictive value of 77.2% within the full study sample. Model 7 demonstrated a Cox

and Snell R2 value of 0.35.


an obtained statistical power of 0.99 for Model 7. A summary of the eight steps used to create

Model 7, including correct classification rates, χ2 change scores, significance of χ2 change scores,

Cox and Snell R2 values, and Cox and Snell R2 change scores for each step can be found in Table

17. Table 18 outlines the predictor variables retained in each of the 8 models along with their

accompanying significance values. Table 19 shows the correct classification rate, sensitivity,

specificity, positive predictive value, negative predictive value, and R2 value for Models 1b

through 7 in the groups from which they were created.


Table 17

Summary of the Eight Steps Completed During Creation of Model 7


change β Exp(β)

Step 1 30.31 <0.01 0.15



Skilled 2.71 14.95

Step 2 16.24 <0.01 0.22 0.07



Skilled 2.61 13.65


Step 3 2.44 0.12 0.23 0.01



Skilled 2.43 11.33


Education 0.15 1.17

Step 4 19.01 <0.01 0.30 0.07



Skilled 2.30 9.97


Education 0.11 1.12

OTBM 0.14 1.15

Step 5 3.61 0.06 0.31 0.01



Skilled 2.40 11.07


Education 0.11 1.11

OTBM 0.17 1.19

COWAT -0.05 0.95


Table 17 (Cont.)


change β Exp(β)

Step 6 3.39 0.07 0.32 0.01



Skilled 2.66 14.29


Education 0.11 1.12

OTBM 0.24 1.27

COWAT -0.05 0.95

RCFT-Immediate -0.03 0.97

Step 7 3.37 0.07 0.34 0.02



Skilled 2.78 16.09


Education 0.10 1.10

OTBM 0.21 1.23

COWAT -0.05 0.95



Step 8 3.25 0.07 0.35 0.01



Skilled 3.03 20.66


Education 0.07 1.07

OTBM 0.21 1.23

COWAT -0.06 0.94





step of model creation. COWAT = Controlled Oral Word Association Test; RCFT-

Immediate = Rey Complex Figure Test immediate recall trial; RCFT-Recognition = Rey

Complex Figure Test recognition trial.


Table 18

Variables Retained During Regression Analyses in Models 1 through 7

Variable Name Model

1

Model

1b

Model

2

Model

3

Model

4

Model

5

Model

6

Model

7

Age 0.04

Education 0.11 0.02 0.01 0.52

Independent driving status 0.09 0.04 0.06 0.09

Occupation (semi-skilled) 0.01 <0.01 <0.01 <0.01

Occupation (skilled) 0.01 <0.01 <0.01 <0.01

Months since injury 0.04 0.13 0.09

PIQ 0.02 0.03 0.01 0.15

COWAT* 0.12 0.07 0.04

Judgment of Line Orientation* 0.05 0.12

Finger Tapping dominant* 0.03 0.03 0.02

Booklet Category* 0.10

RCFT immediate recall* 0.05

RCFT delayed recall* 0.17 0.04

RCFT recognition* 0.05 0.12 0.06

OTBM <0.01 <0.01 <0.01 0.02 <0.01

Note: For each variable in each model, the numbers shown represent the value of p. An

asterisk (*) indicates that the score was represented by a T-score value. Occupation =

premorbid occupation; PIQ = Performance Intelligence Quotient; COWAT = Controlled Oral

Word Association Test; RCFT = Rey Complex Figure Test.


Table 19

Statistical Performance of Models 1b through 7

Model Correct Classification Rate Sensitivity Specificity PPV NPV R2

1b 83.3% 93.8% 61.3% 83.6% 82.6% 0.28

2 86.5% 95.4% 67.7% 86.1% 87.5% 0.36

3 88.5% 98.5% 67.7% 86.5% 95.5% 0.40

4 81.6% 84.4% 77.4% 84.4% 77.4% 0.39

5 80.3% 88.9% 67.7% 80.0% 80.8% 0.36

6 78.6% 91.7% 56.3% 78.2% 80.0% 0.32

7 79.2% 89.3% 62.0% 80.0% 77.2% 0.35

Note: All values represent the statistical performance of each model in the group from which

it was created. PPV = positive predictive value; NPV = negative predictive value.


Discussion

A Parsimonious Model to Predict Employment Status

Several hierarchical regression models were created to develop a parsimonious model

that utilized scores from a standard administration of the MNB to predict employment status in a

population of individuals who have incurred a TBI. The first model included the OTBM as a

single predictor variable in addition to the constant. The second model was identical to the first

with the addition of the Judgment of Line Orientation T-score as a second predictor variable,

although the addition of this variable resulted in a nonsignificant χ2 change score as illustrated in

Table 8. The model was named Model 1 due to the fact that it was created to accomplish Goal 1.

The initial plan for analysis was that model creation would be halted at this point. However, due

to the exploratory nature of this study, model creation was continued. This resulted in the

creation of three additional hierarchical regression models, as shown in Table 9. The first

included the dominant hand T-score from the finger tapping test, and the second included the

Ward 7-subtest WAIS-III PIQ score. The last model omitted the OTBM due to low statistical

significance following the inclusion of additional predictor variables. The final model was

named Model 1b so that it could be distinguished from Model 1 during subsequent comparisons.

It included three variables in addition to the constant: the Ward 7-subtest PIQ, the Judgment of

Line Orientation T-score, and the Finger Tapping Test T-score from the dominant hand. Model

1b was considered the more robust model due to its inclusion of additional predictor variables

and its significant chi-square change score when compared to Model 1, and it was therefore used

in all subsequent analyses.

There appears to be consistency between the predictor variables that were retained in

Model 1b. The Ward 7-subtest PIQ is a measure of general cognitive functioning that primarily


quantifies what the WAIS-III refers to as perceptual organization and processing speed (The

Psychological Corporation, 2002). The Judgment of Line Orientation T-score is another

instrument that is thought to provide a measure of perceptual organization, similar to the

measures that contribute to the PIQ score. The final composition of Model 1b suggests that

measures of processing speed and perceptual organization accounted for a significant amount of

variance in employment status when compared to the other cognitive domains that were

considered during model creation. Even the OTBM, which represents a mean score for all

neuropsychological measures administered, and which can therefore be considered a more global

measurement of cognitive functioning than the PIQ, was removed from Model 1b after the PIQ,

Finger Tapping, and Judgment of Line Orientation scores were retained. This suggests that

scores from instruments that measure domains other than processing speed, perceptual

organization, and motor performance did not significantly account for variance when predicting

employment status.

It is notable that neither Model 1 nor Model 1b retained any measures of memory

performance as significant predictors of employment status following TBI, even though memory

problems are the most often cited and most frequently researched consequence of brain injury

(Vakil, 2005). The amount of time since injury also failed to account for a significant amount of

variance in employment status following TBI in both Model 1 and Model 1b. This suggests that

for participants in Group 1, cognitive performance was more strongly associated with returning

to work than was the amount of time since injury.

Also of note is the R2 value for Model 1b, which remained modest at 0.28. This means

that despite the consideration of neuropsychological test performance and time since injury,

roughly 70% of the variance in employment status in Group 1 was not accounted for. This


finding indicates a wide range of factors not accounted for by Model 1b that influenced the

employment status of participants in Group 1. It is difficult to compare the observed R2 value for

Model 1b with previously existing models, as R2 values have not been consistently reported in

the literature.

Sensitivity, Specificity, and Predictive Efficacy of a Parsimonious Model

Model 1b showed an 83.3% correct classification rate, 93.8% sensitivity, and 61.3%

specificity in Group 1. It also showed positive and negative predictive values of 83.6% and

82.6%, respectively (see Table 12).

The correct classification rate obtained in this study is slightly higher than previous

models that have used neuropsychological assessment and demographic variables to predict

employment outcomes following TBI. Prior studies obtained correct classification rates of

between 65% and 77% (Drake et al., 2000; Fleming et al., 1999; Guerin et al., 2006; Kreutzer et

al., 2003; MacMillan et al., 2002; Simpson & Schmitter-Edgecombe, 2002). While differences in

patient samples and assessment methods make it difficult to make direct comparisons between

studies and the predictor models examined, results from this study suggest that estimates of

employment status following TBI made using measures taken from a standard administration of

the MNB result in higher correct classification rates than those achieved by previous models.

The higher correct classification rates found in this study are important given the fact that the

average MNB administration takes approximately two and a half hours, making it a more

parsimonious instrument than most traditional neuropsychological assessment batteries, which

can last an average of 8 hours (Rabin, Barr, & Burton, 2005). The use of a battery lasting less

than half of the time spent on an average neuropsychological assessment could increase access to


services and reduce overall cost to the patient without sacrificing the accuracy of model

predictions of employment status following a TBI.

The calculated sensitivity and specificity of Model 1b demonstrated an overall

classification rate that favored the correct identification of participants who were employed at the

time of neuropsychological assessment. As shown in Table 12, the sensitivity was found to be

93.8%, while the specificity of Model 1b was much lower, correctly identifying only 61.3% of

unemployed participants as unemployed. Part of the discrepancy between the sensitivity and

specificity of Model 1b could be due to the demographic composition of Group 1. Almost 70%

of the participants in Group 1 were employed at the time of neuropsychological assessment. In

such a case, a model that favored the correct classification of participants as employed, would

achieve a higher overall correct classification rate.

The positive and negative predictive values for Model 1 in Group 1 were nearly equal at

83.6% and 82.6%, respectively. This means that out of 100 people that the model identified as

employed, 83 of them actually were employed. Similarly, out of 100 people that the model

identified as unemployed, 82 of them actually were unemployed. In other words, there is

approximately a 20% chance that each model prediction is incorrect.

The clinical implications of these values vary by individual, and are influenced partly by

the potential risk of harm in resuming a particular type of employment prior to the return of

premorbid neuropsychological functioning. For instance, it may create a safety risk to return a

patient to work prematurely if they operate heavy equipment or work in a hazardous

environment. In other cases, it may be detrimental to an individual’s career to wait too long

before returning to work due to time missed or experience lost.


The Addition of Demographic Predictors

Model 2 was created in order to determine the extent to which demographic variables

could account for variance in employment status in Group 1 beyond that already accounted for

by the neuropsychological assessment measures included in Model 1b. The model was created

by adding the identified demographic variables into a second block of analysis after the

calculation of Model 1b was complete. The new model retained only two demographic variables:

self-reported independent driving status and premorbid occupation (see Table 10). As described

in the Results section above, χ2 goodness of fit calculations showed that the addition of these two

demographic variables added significantly to the amount of variance in employment status

accounted for by Model 1b, from an R2 value of 0.28 to an R2 value of 0.36. Model 2 had only a

slightly higher correct classification rate than Model 1b at 86.5% compared to 83.3%. Model 2

also showed increases in sensitivity, specificity, positive predictive power, and negative

predictive power when compared to Model 1b. While the increases in statistical performance

described above are modest, the inclusion of these measures in predicting employment status

following a TBI is supported by the ease of collecting such information.

The fact that independent driving status was retained as one of two demographic

variables in Model 2 supports the use of this variable as a clinically useful indication of good

overall adjustment and occupational functioning following TBI. This could simply be due to the

fact that individuals who are driving independently are more able to deliver résumés, attend

interviews, and secure transportation to and from work. However, Rapport, Hanks, and Bryer

(2006) found that the amount of social support provided by significant others accounted for more

variance in post-TBI driving status than injury severity, negative affectivity, overall level of

social support, and use of public transportation. These findings suggest that in the current study,


each participant’s independent driving status may be directly related to the amount of social

support they receive from significant others, a variable which was not included in the current

analysis.

An alternate explanation for the retention of independent driving status as a predictor

variable in Model 2 is that driving involves many cognitive functions, such as the maintenance of

divided attention, route planning, muscle coordination, and rapid information processing, all of

which may be necessary to secure and retain successful employment. This explanation is

supported by a follow-up study by Rapport, Bryer, and Hanks (2008) who found that individuals

who were not driving following a TBI produced significantly lower scores on a short

neuropsychological battery than those who were driving following a TBI. The relationship

between driving following a TBI and good cognitive functioning as measured by

neuropsychological assessment has also been highlighted in other studies (D’apolito,

Massonneau, Paillat, & Azouvi, 2013; Labbe, Vance, Wadley, & Novack, 2014). When

combined with the research findings described above, the current study supports the continued

use of independent driving status as a good measure of neuropsychological functioning

following TBI.

Premorbid occupation was retained as an additional predictor variable in Model 2. As

stated above in Data Reduction, the variable was defined categorically, and each participant’s

premorbid occupation was labeled as unemployed, semi-skilled, or skilled. The odds ratios for the

categories semi-skilled and skilled represent the increase in odds that a participant who held

semi-skilled or skilled premorbid employment was employed at the time of neuropsychological

assessment, when compared to participants who were unemployed prior to TBI. The observed

odds ratios for the categories semi-skilled and skilled were very similar, at 12.19 and 12.01


respectively (see Table 10). An odds ratio represents the odds that a given outcome will occur

when a specific criterion is met, compared to the odds of the same outcome will occur if the

criterion is not met. In the case of the current example, the odds of being employed at the time of

neuropsychological assessment were 12 times greater for participants who held semi-skilled or

skilled employment prior to sustaining a TBI than for participants who were not employed prior

to their injury. Given the fact that the odds ratios are similar for both premorbid employment

categories, it is reasonable to conclude that those participants in Group 1 who were working

before sustaining a TBI are more likely to be working after sustaining a TBI, regardless of the

type of employment they held. While this result seems intuitive, it supports the idea that the

occupational impact of sustaining a TBI may be more severe for unemployed individuals than it

is for those who are employed at the time of injury. The retention of premorbid employment as a

predictor variable in Model 2 may also suggest the presence of numerous premorbid, mediating

variables that impact an individual’s ability to secure employment both before and after

sustaining a TBI.

The Additive Value of Neuropsychological Variables

Model 3 was created in order to determine the extent to which neuropsychological

measures could account for variance in employment status in Group 1 beyond that accounted for

by demographic variables alone. The importance of this model lies in the overall cost, both in

time and money, of a full neuropsychological assessment and the relative ease of collecting

demographic information. If employment status can be predicted equally well using demographic

information or neuropsychological measures, then the collection of demographic information

would be a cost-effective alternative to neuropsychological assessment when predicting

employment status (Yates & Taub, 2003).


The model was created by entering variables in two separate blocks, with demographic

variables entered before neuropsychological assessment variables. When analysis was complete,

the model retained the same demographic and neuropsychological variables as Model 2 with the

addition of education as a demographic variable and the COWAT T-score as a

neuropsychological assessment variable, although neither of them reached statistical significance

at the p = 0.05 level (see Table 11). Hosmer and Lemshow tests showed a poor fit for the model

(p = 0.01). These results suggest that model predictions may have been unevenly distributed

when compared to observed employment status in Group 1. However, any unevenness in the

distribution of model predictions was considered to have minimal effect on model performance

due to observed R2 values that were comparable to other models created during this study.

As described in the Results section above, χ2 goodness of fit calculations showed that the

addition of neuropsychological assessment variables added significantly to the amount of

variance in employment status accounted for by the demographic variables alone. Together with

the finding that demographic variables contributed significantly to the amount of variance

accounted for by neuropsychological assessment variables alone during the creation of Model 2,

these results demonstrate that there is statistically significant benefit in using both demographic

and neuropsychological assessment variables together when predicting employment outcomes

following TBI and that any model wishing to predict employment status following TBI should

consider both variable categories.

Comparisons between Models 1b, 2, and 3 show that Model 3 had a slightly higher

correct classification rate than Models 1b and 2 at 88.5% compared to 83.3% for Model 1b and

86.5% for Model 2. The R2 value for Model 3 (R2 = 0.40) was slightly higher than the R2 values

for Models 1b (R2 = 0.28) and 2 (R2 = 0.36). Model 3 also showed increases in sensitivity,


positive predictive power, and negative predictive power when compared to both Models 1b and

2. Most notably, Model 3 demonstrated 98.5% sensitivity in Group 1 while maintaining 67.7%

specificity. This represents a 3.1% increase in sensitivity with no change in specificity when

compared to Model 2, and thus better overall predictive accuracy. For a complete explanation of

the performance of Model 3 in Group 1, see Table 12.

The increase in R2 value, correct classification rate, sensitivity, positive predictive value,

and negative predictive value seen in Model 3 when compared to Models 1b and 2 is most likely

due to the addition of two new predictor variables: education and the COWAT T-score. The

variable education showed an odds ratio of 1.44, indicating that for participants in Group 1, the

odds of being employed at the time of neuropsychological assessment increased by 0.44 with

each additional year of formal education. This effect may be related to the idea of cognitive

reserve, the idea that individuals whose brains process information more efficiently may be

better able to compensate for small deficits in cognitive performance (Stern, 2002). It has been

proposed that increasing levels of education may result in more efficient information processing

and thus a larger cognitive reserve, subsequently reducing the amount of time needed to recover

from a TBI (Schneider et al., 2014). In this case, participants who were more highly educated

would have had more ease in locating employment following a TBI due to the fact that minor

cognitive problems would result in less observable change in their workplace performance. Such

a relationship has been demonstrated by Levi, Rassovsky, Agranov, Sela-Kaufman, and Vakil

(2013), who used structural equation modeling to create a three-factor model of cognitive reserve

in victims of TBI that identified intelligence, socioeconomic status, and involvement in leisure

activities as significant factors.


An alternate explanation for the inclusion of education as a predictor variable in Model 3

can be found in the previously-cited study conducted by Machamer, Temkin, Fraser, Doctor, and

Dikmen (2005), who found that in their sample, older participants were more likely to return to

stable employment following a TBI than younger participants. The authors concluded that this

was due to the fact that older individuals may be more established in their careers and therefore

more highly valued by their employers, who would in turn be willing to let them return to

employment following an injury. Similarly, individuals with more work experience may possess

skill sets that are less easily replaced and would therefore be more highly encouraged to return to

work following an injury. These observations could also explain the relationship between

education and employment status in Model 3, due to the fact that a participant with more formal

education may be more likely to possess valued skill sets, qualifications, or degrees, which

would in turn make him or her less easily replaced by employers and more competitive for

renewed employment following a TBI.

Also included in Model 3 as a predictor variable was the COWAT T-score. The exact

reason why this variable would be retained in Model 3 but not in Model 2 is unclear, as all other

neuropsychological assessment predictor variables are identical in both models. However, the

fact that it was retained in Model 3 supports earlier claims that the test is a sensitive measure of

brain dysfunction and may be significantly correlated with functional impairment (Lezak,

Howieson, and Loring, 2004).

An Investigation of Model Performance

Proportion of variance accounted for. Decreased model performance was expected

when the regression models created in Group 1 were confirmed in Group 2. Confirmatory and

cross-validation analyses of models routinely results in decreased performance due to the fact


that initial data analyses take advantage of chance correlations that are less likely to appear in a

separate sample. As expected, the R2 values for Models 1b, 2, and 3 were consistently higher in

Group 1 than in Group 2. For each model, the between-groups difference in R2 values was

approximately 0.10 (actual differences ranged from 0.08 to 0.11; see table 12). This suggests that

group membership accounted for approximately 10% of the variance in employment status

accounted for by each of the above-mentioned models. It also suggests that when applied to

independent samples, the demographic and neuropsychological variables retained by Models 1b,

2, and 3 may account for less variance in employment status than they did for participants in

Group 1.

Percentage of cases correctly classified. As illustrated in Table 12, Models 1b, 2, and 3

showed lower percentages of cases correctly classified in Group 2 than in Group 1. When

comparing model performance in Group 1 and Group 2, the between-group differences in the

percentage of cases correctly classified were similar for all three models, ranging from 13.5% to

16.6%. Model 3, which demonstrated the highest correct classification rate in Group 1, showed

the largest difference and no longer demonstrated the highest correct classification rate in Group

2. This could be due in part to the fact that Model 3 retained three additional non-significant

predictor variables that were not included in Model 2. Given the fact that these variables were

non-significant in Group 1, they may have accounted for even less variance in employment

status in Group 2 and therefore had a negative impact on the accuracy of predictions.

From a clinical perspective, the between-group differences described above may be

important, as they represent the incorrect classification of approximately 13 to 16 additional

individuals when Models 1b, 2, and 3 were applied to an independent sample. These results are

consistent with the finding that R2 values were consistently lower in Group 2, and support the


idea that the statistical performance of Models 1b, 2, and 3 is noticeably impacted by group

membership. However, the correct classification rates of all three models in Group 2 were still

comparable to the correct classification rates of previous models (Drake et al., 2000; Fleming et

al., 1999; Guerin et al., 2006; Kreutzer et al., 2003; MacMillan et al., 2002; Simpson &

Schmitter-Edgecombe, 2002).

Sensitivity. Table 12 shows that the sensitivity of Models 1b, 2, and 3 was lower in

Group 2 than it had been in Group 1. The sensitivity of Models 1b, 2, and 3 in Group 2 ranged

from 83.9% to 87.5%. This means that approximately 84% to 88% of participants in Group 2

who were working at the time of neuropsychological assessment were correctly identified as

such by the 3 models. These outcomes suggest that despite lower sensitivity when applied to an

independent sample, all three models still retain clinical utility when trying to determine whether

or not a particular patient is ready to return to work. While Model 2 showed the highest

sensitivity in Group 2, any between-model differences are considered to be negligible.

Specificity. Table 12 shows that the specificity of Models 1b, 2, and 3 was lower in

Group 2 than it had been in Group 1. On average, the between-group differences in specificity

for each model were greater than the between-group differences in sensitivity. The specificity of

Models 1b, 2, and 3 in Group 2 ranged from 47.5% to 55.0%, indicating that in some cases the

models correctly identified less than half of the participants in Group 2 who were not working at

the time of neuropsychological assessment. However, as described above under percentage of

cases correctly classified, overall predictive accuracy was comparable to previous prediction

models, thus demonstrating the continued utility of Models 1b, 2, and 3 in Group 2.

Positive Predictive Value. The positive predictive value of Models 1b, 2, and 3 in Group

2 ranged from 69.6% to 72.3%, with Model 1b demonstrating the lowest positive predictive


value and model 3 demonstrating the highest. This means that of all the participants identified by

each model as being employed at the time of neuropsychological assessment, approximately

70% of them were actually employed at that time. The clinical utility of a positive predictive

value of approximately 70% will depend on the occupational setting, and the specific risk

involved in returning an individual to work too early or too late.

Subjective between group comparisons show that these values are lower in Group 2 than

they had been in Group 1, where the models demonstrated positive predictive values ranging

from 83.6% to 86.5%.

Negative Predictive Value. The negative predictive value of Models 1b, 2, and 3 in

Group 2 ranged from 70.4% to 75.0%, with Model 1b showing the lowest negative predictive

power and Model 2 showing the highest. This means that of all the participants in Group 2 that

were classified as unemployed at the time of neuropsychological assessment, between 70% and

75% of them were actually unemployed at that time, depending on the specific prediction model

used. As was the case with positive predictive value, the clinical utility of the observed negative

predictive power will depend on the specific setting in which each individual is employed.

Subjective between group comparisons show that the negative predictive value of each

model was lower in Group 2 than it had been in Group 1.

In general, between-group comparisons showed that Models 1b, 2, and 3 all demonstrated

reduced performance in Group 2 on all identified performance outcome measures. Any attempt

to generalize the findings of this study to other populations should take into consideration the

fact that model performance is reduced in independent samples. However, in most cases these

differences were not large enough to preclude clinical utility.


The Relationship between Litigation Status and Model Performance

Upon initial examination, the performance of Models 1b, 2, and 3 appears to be lower in

litigants than in non-litigants (see Table 13). As a general rule, all three models demonstrated

lower performance on identified outcome measures in Group 3 than they had in Group 4.

However, closer inspection shows that the sensitivity and negative predictive power of all three

models was higher in the group of litigants than in the group of non-litigants. This observation is

likely the result of a low unemployment rate in Group 3, in which only 9% of the sample was

unemployed at the time of neuropsychological assessment. As explained above, all three models

demonstrated moderate levels of specificity in Groups 1 and 2. However, in the group of

litigants, which consisted of only 7 participants who were unemployed as opposed to 69

participants who were employed at the time of assessment, the few participants who were

classified as unemployed would represent almost the entire cohort of unemployed participants.

Despite any differences in model performance, overall correct classification rates in both

Groups 3 and 4 were comparable to the correct classification rates reported in earlier studies.

Also of note is the fact that the R2 values for Models 1b, 2, and 3 were nearly identical in Groups

3 and 4, with an average between-groups difference of 0.02. This suggests that there is no

clinically significant difference in the amount of variance accounted for by the

neuropsychological and demographic variables retained by Models 1b, 2, and 3 when comparing

litigants and non-litigants. The combination of these findings suggests that all three models can

be applied in both populations with subjectively similar results.

Model Creation in a Sample of Litigants

When Models 4 and 5 were created in Group 3, there were several differences in the

predictor variables that were retained when compared to Models 1b, 2, and 3 (see Tables 14 and


15). Perhaps most notably, the number of months since injury was retained by both models, and

accounted for a significant (p = 0.04) amount of the variance in employment status in Model 4.

In both models, participants were more likely to be employed when more time had passed since

their injury. The fact that this variable was retained in Models 4 and 5 but not in Models 1b, 2, or

3 suggests that the number of months since injury explains a higher proportion of variance in

employment status for individuals who are involved in litigation at the time of their assessment.

There are several reasons why this difference might appear. One reason is that a more

severe injury would result in more severe cognitive deficits, thus increasing both recovery times

and the chances of being involved in litigation. However, this does not seem to be the case in the

current sample, since the results of between-group MANOVA analysis of neuropsychological

assessment variables did not reach significance (p = 0.76), indicating that litigants in Group 3

and non-litigants in Group 4 were not significantly different on all global measures of cognitive

functioning including the WAIS-III FSIQ, WAIS-III PIQ, WAIS-III VIQ, and OTBM. These

findings indicate that between group differences on the neuropsychological and demographic

variables of interest did not exceed those expected by chance, and that cognitive impairment

following injury was similar in both groups.

Another likely scenario is that the number of months since injury accounts for a

significant amount of variance in employment status in Group 3 as a direct result of the litigation

process itself. Individuals who are involved in litigation resulting from a head injury, and

especially those who are making disability claims, may be suspected of malingering if they

return to work while their cases are being heard. As a result, they would be required to wait until

their case has been settled before returning to work, and individuals assessed soon after their

injury would be less likely to have resumed employment.


Another notable way in which Models 4 and 5 differ from Models 1b, 2, and 3 is that the

OTBM, which was not retained in any of the first three models, was highly significant in both

Model 4 (p < 0.01) and Model 5 (p < 0.01). The Rey Complex Figure Test delayed recall trial

was the only other neuropsychological assessment variable that was retained by both Models 4

and 5. This would suggest that while litigants did not necessarily show more severe impairment

in any specific neuropsychological domain, they might have shown a more diffuse pattern of

deficits than those who were not involved in litigation. This finding is interesting in that it

suggests that individuals involved in litigation following a TBI may experience a wider range of

minor deficits that prevent them from returning to work, as opposed to severe deficits in any one

specific neuropsychological domain.

With only one exception, Models 4 and 5 showed lower R2 value, percentage of cases


value in Group 3 than Models 1b, 2, and 3 did in Group 1. This seems to suggest that, given the

demographic and neuropsychological assessment variables used in this study, it is more difficult

to accurately predict the employment status of individuals when they are involved in litigation.

Models 4 and 5 may have both benefited from the inclusion of additional predictor variables that

are more directly related to the litigation itself, such as whether the litigation involved a

disability claim or whether the participant had been assigned public or private legal

representation.

Model Creation Using the Entire Study Sample

When the entire study sample was used to create Models 6 and 7, the variables retained

were very similar to those retained in previous models. Only one variable retained by Model 7,

the Rey Complex Figure Test immediate recall trial T-score, had not been retained by any of the


previous models. Although the implications of retaining this variable are unclear, its inclusion

suggests that with larger sample sizes and the resulting increase in statistical power, models may

be more able to identify variables that account for significant amounts of variance in

employment status following TBI. Future research would benefit from the use of larger sample

sizes.

Also notable was the highly significant contribution of the OTBM in both Model 6 (p =

0.02) and Model 7 (p < 0.01). No other neuropsychological assessment variable attained such

high statistical significance in any of the models created in this study. In Step 1 of Model 6, the

inclusion of the OTBM alone resulted in an R2 value of 0.21, nearly equivalent to the R2 value of

Model 1b in Group 1 (R2 = 0.28). This finding demonstrates that with increased sample sizes and

the resulting increase in individual variation across neuropsychological performance, the

importance of global indicators of cognitive performance in the prediction of employment status

is increased. This finding also demonstrates the utility of the OTBM in predicting employment

status following a TBI, and supports its continued use as an outcome measure in TBI research.

The observed utility of global indicators of cognitive performance in predicting

employment status can also help inform treatment recommendations that might increase the

chances of a successful return to work following TBI. Cicerone, Mott, Azulay, Sharlow-Galella,

Ellmo, Paradise, and Friel (2008) tracked the functional outcomes of 68 adults with mild to

severe TBI as they participated in either a traditional neuropsychological rehabilitation program

that focused on the remediation of specific cognitive deficits, or a comprehensive holistic

neuropsychological rehabilitation program that focused on the improvement of general

neuropsychological performance, metacognition, interpersonal functioning, and emotional

regulation. After 16 weeks of treatment, participants from the holistic treatment program showed


better community integration and quality of life as measured by the Community Integration

Questionnaire (effect size = 0.59) and the Perceived Quality of Life Scale (effect size = 0.30).

Geurtsen, van Heugten, Martina, and Geurts (2010) reviewed 13 neuropsychological

rehabilitation studies published between 1990 and 2008 and found that comprehensive

neuropsychological rehabilitation programs significantly improved community integration

following severe TBI. This finding was supported by Cicerone and colleagues (2011), who

reviewed 141 cognitive rehabilitation studies published between 2003 and 2008 and found strong

evidence for the use of comprehensive, holistic neuropsychological rehabilitation programs

following mild to severe TBI. Combined with these findings, the current study supports the use

of comprehensive rehabilitation programs that focus on the treatment of a broad range of

neuropsychological and functional deficits following TBI rather than programs which focus on

addressing specific neuropsychological domains.

Interestingly, the percentage of cases correctly classified, sensitivity, specificity, positive

predictive value, and negative predictive value of Models 6 and 7 in the study sample were

consistently lower than the same values for Models 1b, 2, and 3 in Group 1. While the reasons

for this finding are not entirely clear, it may be partly due to the fact that in logistic regression

smaller sample sizes can lead to overly optimistic estimates of model performance (Leeflang,

Moons, Reitsma, & Zwinderman, 2008), partly due to an artificial inflation of odds ratios

(Bohning, Holling, & Patilea, 2010). Therefore, the decreased value of statistical performance

outcomes for Models 6 and 7 when compared to Models 1b, 2, and 3, can be attributed in part to

the increase sample size during model creation.


The Difficulty of Predicting Employment Status

The current study examined the degree to which data from a standard MNB

administration correctly predicted the current employment status of individuals who had

previously incurred a TBI. The correct classification rates of the models created in this study

ranged from 78.6% to 88.5%. This is consistent with the predictive efficacy of similar models

from previous research that demonstrated correct classification rates ranging from 65% to 77%

(Drake, Gray, Yoder, Pramuka, & Llewellyn, 2000; Fleming, Tooth, Hassell, & Burchan, 1999;

Guerin, Kennepohl, Leveille, Dominique, & McKerral, 2006; Kreutzer et al., 2003; MacMillan,

Hart, Martelli, & Zasler, 2002; Simpson & Schmitter-Edgecombe, 2002). Thus, we can conclude

that the models created using data from a standard MNB administration were able to correctly

predict employment status as well as models created using a variety of other neuropsychological

measures. However, R2 values of the models created in this study ranged from 0.29 to 0.40. The

fact that only 30% to 40% of variance in employment status was accounted for highlights the

difficulty in predicting employment status following a TBI, and suggests that additional variables

should be considered in order to increase predictive accuracy. In fact, employment is a highly

complex variable that can be influenced by a wide range of individual characteristics and aspects

of the environment that might not be measured by a neuropsychological battery. Skills such as

being able to accurately describe past work experience and being able to accurately evaluate your

own strengths, as well as being able to demonstrate proper social skills and situational judgment

may impact performance during a job interview (Salgado & Moscoso, 2002). Even an

applicant’s accent may affect perceived employability (Rakic, Steffens, & Mummendey, 2011).

These same constructs may also affect the likelihood that an individual will maintain

employment once they have been hired.


While many of these skills are related to neuropsychological constructs, a traditional

neuropsychological assessment may lack the ecological validity needed to predict performance

in a situation as complex as a job interview (Chaytor & Schmitter-Edgecombe, 2003). For

example, scores from neuropsychological tests designed to measure verbal fluency would be

expected to share some variance with an individual’s overall communication skills. However, the

ability to perform well on a test of verbal fluency may not be sufficient to succeed in a job

interview, where communication is highly complex and the environment is less controlled and

predictable. Behavioral assessment measures such as the Frontal Systems Behaviour Scale have

been shown to be more effective than traditional neuropsychological assessment in predicting

community integration following TBI (Reid-Arndt, Nehl, & Hinkebein, 2007). The addition of

similar behavioral variables into a model meant to predict employment status may increase

correct classification rates and variance accounted for.

Even a model that accounts for a comprehensive spectrum of behavioral, psychological,

and cognitive performance variables might only have moderate success in predicting

employment status following a TBI. Employment status is affected by a wide range of

environmental variables that are largely unrelated to the individual being tested. These variables

could include the rate at which individuals are entering and exiting the workforce (Shimer,

2012), wage rigidity (Haefke, Sonntag, & van Rens, 2013) the distribution of available jobs

across large and small employers (Moscarini & Postel-Vinay, 2012), the presence of secondary

gain (Harris, Mulford, Solomon, van Gelder, & Young, 2005; Schneider, Bassi, & Ryan, 2011),

and government tax policies (Arnold, Brys, Heady, Johansson, Schwellnus, & Vartia, 2011). All

of these variables would affect an individual’s ability to find work regardless of whether or not

he or she had sustained a TBI and independently of their ability to perform well on a


neuropsychological assessment battery. At times when economic conditions are particularly

poor, these environmental factors may have an increased influence on an individual’s ability to

find employment following a TBI due to the fact that the market may become saturated with

highly qualified applicants competing for a limited number of positions. Conversely,

environmental factors may also account for increased variance in employment rates following a

TBI when economic conditions are highly favorable. In such a case, an increase in the number of

jobs available may reduce the effect that of cognitive abilities and limitations have on the ability

to find employment.

Limitations of the Current Study

The current study faced a number of limitations. As is the case with all studies using

archived data sets, the variables included for analysis were limited to those that had been

collected prior to the creation of this research design. While efforts were made to include the

same measures that had been included in previous investigations of employment outcomes

following TBI, this was not always possible. In many cases, neuropsychological instruments

were included which were thought to be conceptually similar to, or which were thought to

provide measures of similar constructs as the instruments used in previous studies. However, no

two neuropsychological tests are identical. Each measures a unique component of the construct

of interest and is associated with unique sources of measurement and inferential error. In the

current study it is difficult to determine the extent to which the use of novel but conceptually

similar assessment instruments affected the performance of the prediction models, or if

predictive efficacy would have improved if different instruments had been used for data

collection. A similar statement can be made for demographic variables, which may have been


defined or quantified differently in this study than they had been in prior research due to the

limitations established by data collection methods.

Perhaps the greatest limitation of the current study was the homogeneity of the sample

demographics. The study originally planned to include ethnicity as a demographic variable

containing seven distinct categories. The decision to include ethnicity as a predictor variable had

been made based on previous research showing that an individual’s ethnicity was predictive of

employment outcomes following a TBI even after other demographic and injury-characteristic

variables had been controlled for (Arango-Lasprilla et al., 2009). However, the database sample

used in the current study was predominantly White, leaving the other six planned categories of

ethnicity with insufficient membership to be included in analysis. In an attempt to measure the

impact of ethnic status, six of the categories of ethnicity were combined to create a single ethnic

minority category. However, group membership in the ethnic minority category was still

insufficient for inclusion in analysis and a decision was made to not include the variable in the

final analysis.

While the models created during this study showed higher correct classification rates than

similar models created in previous research studies, the fact that ethnicity was not considered by

any of the final prediction models is considered a major limitation. The results of the current

study can only be considered applicable to a primarily White population, and the extent to which

these results might generalize to other populations is unknown. Any attempt to apply the results

of this study to members of ethnic minority groups should be undertaken with caution, especially

in light of research demonstrating the importance of ethnicity on employment outcomes

following TBI (Arango-Lasprilla, et. al, 2009; Arango-Lasprilla, Ketchum, Lewis, Krch, Gary, &

Dodd, 2011; Forslund, Roe, Arango-Lasprilla, Sigurdardottir, & Andelic, 2013).


This study also included two demographic variables that were reduced prior to data

analysis due to low group membership. The variable premorbid occupation was originally

planned to include seven categories. However, due to low membership in some categories, the

variable was reduced to three groups: not employed, partially employed, and fully employed.

While efforts were made to place the original seven categories in the most appropriate group

following the data reduction, there was often no clear distinction between partial employment

and full employment. For instance, the employment category such as student may have included

participants who were attending school full-time, participants who were attending school part-

time, and participants who were working at another job while attending school. While the

employment category student was included in the category fully employed due to the perceived

neuropsychological demands of full-time education, the division between partial employment

and full employment was a subjective one. With a database that included participants from a

greater variety of occupations, employment categories could be more precisely defined. This

could be important due to the fact that not all full-time employment demands the same cognitive

resources, and impairment in a specific neuropsychological domain might be related to poor

employment outcomes for individuals in certain career fields.

The variable independent driving status was also reduced before inclusion in data

analysis. The variable originally contained four categories: not driving, partially not driving,

partially driving, and driving. Due to low group membership, this variable was reduced to two

categories: not driving and driving. The extent to which the inclusion of all four originally

planned categories may have affected the results of this study is unknown. However, the fact that

numerous variables had to be reduced before analysis due to low membership in certain

categories represents a larger problem. Such cases of low expected frequencies can result in low


statistical power and inflated odds ratios (Tabachnick & Fidell, 2007), and a more diverse

database sample would have resulted in higher statistical power and more accurate results during

analysis.

A similar limitation to this study exists in the reduction of the outcome variable

employment status. The variable was originally meant to include nine possible categories, seven

of which represented civilian employment categories and two of which represented military

employment categories. For the purposes of the current study, these employment categories were

originally collapsed into three main employment outcome groups based on the estimated

neuropsychological demand of each category. The three employment outcome groups were not

employed, partially employed, and fully employed. However, due to low membership in the

partially employed category, the variable was reduced again to contain only the categories

employed and unemployed. This reduction resulted in the need to change the planned statistical

analysis to a binary regression meant to predict whether or not each participant was employed

following a TBI. However, the change in statistical analyses eliminated the ability to predict

whether or not subjects were employed in the same type of employment that they held before

their injury, or if they had returned to a position that had fewer cognitive demands or potentially

paid a lower salary. In fact, before the variable employment status had been reduced, only one

participant in the study sample qualified for inclusion in the fully employed category, suggesting

that a return to premorbid occupations may be very difficult following a TBI, regardless of

neuropsychological functioning. However, due to sample limitations, conclusive statements

about the difficulty of returning to full vs. partial employment following TBI are beyond the

scope of the current study. This information would be very useful to the clinician who is making

recommendations about when a patient might be ready to return to work following a head injury


and who would like to assist the patient in establishing realistic expectations about what type of

employment they might be able to find.

The reduction of the employment status variable also resulted in the loss of the military

employment outcomes full duty and limited duty. The fact that these categories were not included

limits the ability to apply results from this study to a military population due to unique

considerations when working with military members who have sustained a TBI, including the

presence of secondary gain (French, Anderson-Barnes, Ryan, Zazeckis, & Harvey, 2012).

This study was also limited by the fact that no distinction was made between participants

with mild, moderate, and severe TBI. The decision to collapse all three categories of TBI into a

single group was made in order to maximize group membership in the study sample and thus

increase statistical power. However, the inclusion of mild TBI in the same sample as moderate

and severe TBI could be misleading. In many of the prediction models created during this study,

the time since injury at the time of neuropsychological assessment did not account for a

significant amount of variance in employment status. However, other research has shown that the

time since injury is one of the best predictors of outcomes following a mild TBI, with injury

characteristic variables being more predictive of recovery outcomes in moderate and severe TBI

(Iverson, 2005; McCrea, 2008). It is likely that the inclusion of all three categories of TBI in a

single sample has resulted in the non-inclusion of potentially significant predictor variables.

Employment status models created using data only from participants with a history of mild TBI

may find that the time since injury accounts for a more significant amount of variance in

employment outcomes.

A final limitation of this study involves the composition of the comparison groups used to

test the models created during the initial statistical analyses. The comparison groups were created


by stratifying the database sample by age and randomly assigning participants into one of two

groups. This was done in order to demonstrate the external validity and generalizability of the

models created within the first group. However, inferences from the comparisons are limited in

that the comparison groups were taken from the same original study sample and therefore to not

represent truly independent groups. It is expected that model performance in a truly independent

sample would be lower than was demonstrated in this study, especially considering the

limitations presented by the composition of this study’s sample as described above.

Directions for Future Research

The prediction of employment outcomes following a TBI is a topic that is likely to be of

continued interest in the future. The results and limitations described above suggest several

directions for future research. The current study provided valuable insight into the variables that

account for a significant amount of variance in employment status following a TBI, especially

broad measures of neuropsychological performance such as the WAIS-III index scales and the

OTBM. It also demonstrates the utility of data collected from a standard administration of the

MNB in making predictions about an individual’s employment status following a TBI. However,

the applicability of these results to ethnic minorities and members of the military is limited.

Future research should be conducted to include data from a higher percentage of participants

representing ethnic minorities as well as active duty service members. Such research would

provide valuable information about demographic and neuropsychological variables that

consistently account for variance in employment status following a TBI and those variables that

account for variance only in certain populations. It would also provide data regarding the utility

of a standard MNB administration in predicting employment status following TBI across

multiple populations.


Future research would also benefit from the independent investigation of mild TBI as

opposed to moderate and severe TBI. Given previous findings that injury characteristics are

predictive of recovery outcomes mostly in moderate and severe TBI (McCrea, 2008), it is likely

that the separate analysis of data from individuals with mild TBI would provide a more accurate

description of variables that are related to employment status following a mild head injury.

The current research study utilized predictor variables that were the same as, or

conceptually similar to, predictor variables that had been used in previous research studies. This

was done in order to maintain a strong empirical basis for analysis and to facilitate comparison

between the MNB and other, previously investigated, flexible neuropsychological batteries in the

prediction of employment status following TBI. However, the results of this analysis showed the

utility of large, summative predictor variables such as the WAIS-III index scores and the OTBM

in predicting employment outcomes. Future research could combine scores on individual

measures to create large, aggregate variables representing specific categories of

neuropsychological functioning. For instance, z-score composites or mean T-scores could be

calculated using scores from all neuropsychological tests measuring attention to create an

aggregated attention score, which could then be entered as a predictor variable in logistic

regression analyses. The use of such aggregate measures would increase statistical power and

may increase the accuracy of model predictions by allowing the consideration of data from

multiple instruments in a single predictor variable.

Currently, research attempting to predict employment outcomes following TBI has not

completely recognized the complexity of employment status. While many of these studies

recognize the contribution of demographic and neuropsychological variables that may be related

to employment outcomes following a head injury, other potentially significant variables such as


personality style and interpersonal skills have been largely ignored. An argument can be made

that such long-standing character traits would have existed prior to the injury and should

therefore not impact employment status following an injury any more than they did prior to the

injury. However, it is also known that changes in personality characteristics are associated with

injuries to the frontal lobes, and that the frontal lobes are particularly vulnerable to injury in TBI

(Zappala, Thiebaut de Schotten, & Eslinger, 2012). Prediction models that account for

personality traits may be better able to measure the impact of frontal lobe injury on employment

status. Future research would benefit from the inclusion of personality measures and measures of

interpersonal functioning.

The available research on employment outcomes following TBI is also limited in that it

ignores external factors that may impact a person’s ability to find employment following a

serious injury. These external factors could include economic factors such as unemployment

rates and the strength of local and national economies. Lundqvist and Samuelsson (2012)

referred to these external factors as “society factors” (p. 13), and make the point that favorable

society factors must be in place in order for a successful return to work to take place. Future

research would benefit from the inclusion of variables meant to provide measures of these

society factors, such as the general unemployment rate at the time of neurological assessment or

an appropriate measure of the strength of local and national economies. The inclusion of such

variables may help account for additional variance in employment status, increase the accuracy

of model predictions, and improve our understanding of variables that account for variance in

employment outcomes following TBI.

Given the current growth of scientific knowledge and general awareness regarding TBI, it

is likely that the prediction of outcomes following a brain injury will remain highly relevant for


many years to come. The continuation of research regarding employment outcomes following

TBI will prove to be vital to our understanding of individual functioning and quality of life

following an injury, and will improve our ability to provide quality care for individuals who have

experienced a TBI.


References

Acker, M. B., & Davis, J. R. (1989). Psychology test scores associated with late outcome in head

injury. Neuropsychology, 3(3), 123-133.

Adams, R. L., & Trenton, S. L. (1981). Development of a paper-and-pen form of the Halstead

Category Test. Journal of Consulting and Clinical Psychology, 49(2), 298-299. doi:

10.1037/0022-006X.49.2.298

Alvarez, J. A., & Emory, E. (2006). Executive function and the frontal lobes: A Meta-analytic

review. Neuropsychology Review, 16(1), 17-42. doi: 10.1007/s11065-006-9002-x

Andelic, N., Stevens, L. F., Sigurdardottir, S., Arango-Lasprilla, J. C., & Roe, C. (2012).

Associations between disability and employment 1 year after traumatic brain injury in a

working age population. Brain Injury, 26(3), 261-269. doi:

10.3109/02699052.2012.654589

Arango-Lasprilla, J. C., Ketchum, J. M., Gary, K. W., Kreutzer, J. S., O’Neil-Pirozzi, T. M.,

Wehman, P., … Jha, A. (2009). The influence of minority status on job stability after

traumatic brain injury. PM&R: The Journal of Injury, Function, and Rehabilitation, 1(1),

41-49. doi: 10.1016/j.pmrj.2008.07.001

Arango-Lasprilla, J. C., Ketchum, J. M., Lewis, A. N., Krch, D., Gary, K. W., & Dodd, B. A.

(2011). Racial and Ethnic Disparities in Employment Outcomes for Persons With

Traumatic Brain Injury: A Longitudinal Investigation 1-5 Years After Injury. PM&R,

3(12), 1083-1091. doi: 10.1016/j.pmrj.2011.05.023

Arnold, J. M., Brys, B., Heady, C., Johansson, A., Schwellnus, C., & Vartia, L. (2011). Tax

Policy for Economic Recovery and Growth. The Economic Journal, 121(550), F59-F80.

doi: 10.1111/j.1468-0297.2010.02415.x


Arnould, A., Rochat, L., Azouvi, P., & Van der Linden, M. (2013). A Multidimensional

Approach to Apathy after Traumatic Brain Injury. Neuropsychology Review, 23(3), 210 –

233. doi: 10.1007/s11065-013-9236-3

Artman, L. K., & McMahon, B. T. (2013). Functional limitations in TBI and their relationship to

job maintenance following work re-entry. Journal of Vocational Rehabilitation, 39(1),

13-21. doi: 10.3233/JVR-130638

Belanger, H. G., Curtiss, G., Demery, J.A., Lebowitz, B.K., & Vanderploeg, R.D. (2005).

Factors moderating neuropsychological outcomes following mild traumatic brain injury:

a meta-analysis. Journal of the International Neuropsychological Society, 11(3), 215-

227. doi: 10.1017/S1355617705050277

Benton, A., Hamsher, K., Varney N., & Spreen, O. (1983). Contributions to Neuropsychological

Assessment: A Clinical Manual. New York: Oxford University Press.

Benton, A. L., & Hamsher, K. deS. (1989). Multilingual Aphasia Examination (3rd ed.). Iowa

City, IA: AJA.

Benton, A. L., Van Allen, M. W., & Fogel, M. L. (1964). Temporal orientation in cerebral

disease. Journal of Nervous and Mental Disease, 139(2), 110-119. doi:

10.1097/00005053-196408000-00003

Benton, A. L., Varney, N. R., & Hamsher, K. D. (1978). Visuospatial judgment: A clinical test.

Archives of Neurology, 35(6), 364-367.

Bigler, E. D., Farrer, T. J., Pertab, J. L., James, K., Petrie, J., & Hedges, D. W. (2013).

Reaffirmed Limitations of Meta-Analytic Methods in the Study of Mild Traumatic Brain


Injury: A Response to Rohling et al. The Clinical Neuropsychologist, 27(2), 176-214. doi:

10.1080/13854046.2012.693950

Binder, L. M., & Rohling, M. L. (1996). Money matters: A meta-analytic review of the effects of

financial incentives on recovery after closed-head injury. American Journal of

Psychiatry, 153(1), 7-10. Retrieved from: http://ajp.psychiatryonline.org/index.dtl

Bohning, D., Holling, H., & Patilea, V. (2010). A limitation of the diagnostic-odds ratio in

determining an optimal cut-off value for a continuous diagnostic test. Statistical Methods

in Medical Research, 20(5), 541-550. doi: 10.1177/0962280210374532

Bornstein, R. A., Baker, G. B., & Douglas, A. B. (1987). Short-term retest reliability of the

Halstead-Reitan Battery in a normal sample. Journal of Nervous and Mental Disease,

175(4), 229-232. doi: 10.1097/00005053-198704000-00007

Buschke, H. (1973). Selective reminding for analysis of memory and learning. Journal of Verbal

Learning and Verbal Behavior, 12(5), 543-550. doi: 10.1016/S0022-5371(73)80034-9

Cattelani, R., Tanzi, F., Lombardi, F., & Mazzucchi, A. (2002). Competitive re-employment

after severe traumatic brain injury: clinical, cognitive, and behavioural predictive

variables. Brain Injury, 16(1), 51-64. doi:10.1080/02699050110088821

Chamelian, L., & Feinstein, A. (2004). Outcome after mild to moderate traumatic brain injury:

The role of dizziness. Archives of Physical Medicine and Rehabilitation, 85(10), 1662-

1666. doi:10.1016/j.apmr.2004.02.012

Chaytor, N., & Schmitter-Edgecombe, M. S. (2003). The Ecological Validity of

Neuropsychological Tests: A Review of the Literature on Everyday Cognitive Skills.

Neuropsychology Review, 13(4), 181-197. doi: 10.1023/B:NERV.0000009483.91468.fb


Cicerone, K. D., Langenbahn, D. M., Braden, C., Malec, J. F., Kalmar, K., Frass, M.,...Ashman,

T. (2011). Evidence-Based Cognitive Rehabilitation: Updated Review of the Literature

From 2003 Through 2008. Archives of Physical Medicine and Rehabilitation, 92(4), 519-

530. doi: 10.1016/j.apmr.2010.11.015

Cicerone, K. D., Mott, T., Azulay, J., Sharlow-Galella, M. A., Ellmo, W. J., Paradise, S., & Friel,

J. C. (2008). A Randomized Controlled Trail of Holistic Neuropsychologic Rehabilitation

After Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 89(12),

2239-2249. doi: 10.1016/j.apmr.2008.06.017

Cifu, D. X., Keyser-Marcus, L., Lopez, E., Wehman, P., Kreutzer, J. S., Englander, J., & Walter,

H. (1997). Acute predictors of successful return to work 1 year after traumatic brain

injury: A multicenter analysis. Archives of Physical Medicine and Rehabilitation, 78(2),

125-131. doi:10.1016/S0003-9993(97)90252-5

Cope, D. N. (1982). Social service data 3 form: Head injury demographic outcome survey. In S.

Berrol (Ed.), Head Injury Rehabilitation Project Final Report (Vol. I, Appendix G, pp. 1-

3). San Jose, CA: Santa Clara Valley Medical Center.

Coronado, V. G., McGuire, L. C., Sarmiento, K., Bell, J., Lionbarger, M. R., Jones, C. D,…Xu,

L. (2012). Trends in Traumatic Brain Injury in the U.S. and the public health response:

1995-2009. Journal of Safety Research, 43(4), 299-307.

D’apolito, A. C., Massonneau, A., Paillat, C., & Azouvi, P. (2013). Impact of brain injury on

driving skills. Annals of Physical and Rehabilitation Medicine, 56(1), 63-80.

Davis, J. J., McHugh, T. S., Axelrod, B. N., & Hanks, R. A. (2012). Performance Validity and

Neuropsychological Outcomes in Litigants and Disability Claimants. The Clinical

Neuropsychologist, 26(5), 850-865. doi: 10.1080/13854046.2012.686631


de Guise, E., LeBlanc, J., Tinawi, S., Lamoureux, J., & Feyz, M. (2012). Acute Relationship

between Cognitive and Psychological Symptoms of Patients with Mild Traumatic Brain

Injury. ISRN Rehabilitation, 2012, Article ID 147285. doi: 10.5402/2012/147285

Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (1987). California Verbal Learning Test

(Research Edition). San Antonio, TX: Psychological Corporation.

Delis, D. C., Kramer, J. H., Kaplan, K., & Ober, B. A. (2000). California Verbal Learning Test-

Second Edition (CVLT-II). San Antonio, TX: Psychological Corporation.

De Renzi, E., & Vignolo, L. A. (1962). The Token Test: A sensitive test to detect disturbances in

aphasics. Brain, 85, 665-678. doi: 10.1093/brain/85.4.665

Derogatis, L. R. (1994). SCL-R: Symptom Checklist-90-R: Administration, Scoring, and

Procedures Manual (3rd ed.). Minneapolis: National Computer Systems.

Dikmen, S. S., Heaton, R. K., Grant, I., & Tempkin, N. R. (1999). Test-retest reliability and

practice effects of expanded Halstead-Reitan Neuropsychological Test Battery. Journal

of the International Neuropsychological Society, 5(4), 346-356. Retreived from:

http://journals.cambridge.org/action/displayJournal?jid=INS

Dikmen, S. S., Machamer, J. E., Winn, H. R., & Temkin, N. R. (1995). Neuropsychological

outcome at 1-year post head injury. Neuropsychology, 9(1), 80-90. doi: 10.1037/0894-

4105.9.1.80

Drake, A. I., Gray, N., Yoder, S., Pramuka, M., & Llewellyn, M. (2000). Factors predicting

return to work following mild traumatic brain injury: A discriminant analysis. Journal of

Head Trauma Rehabilitiation, 15(5), 1103-1112. Retrieved from: http://journals.lww.com

/headtraumarehab/pages/default.aspx


Ekstrom, R. B., French, J. W., Harman, H. H., & Derman, D. (1976). Map Planning Test:

Manual Kit of Factor-Referenced Cognitive Tests. Princeton, NJ: Educational Testing

Service.

Elias, L. J., & Saucier, D. M. (2006). Neuropsychology: Clinical and Experimental Foundations.

Boston, MA: Pearson Education, Inc.

Farbota, K. D., Bendlin, B. B., Alexander, A. L., Rowley, H. A., Dempsey, R. J., & Johnson, S.

C. (2012). Longitudinal diffusion tensor imaging and neurospychological correlates in

traumatic brain injury patients. Frontiers in Human Neuroscience, 6, article 160. doi:

10.3389/fnhum.2012.00160

Faul, M., Wald, M. M., Xu, L., & Coronado, V. G. (2010). Traumatic Brain Injury in the United

States: Emergency Department Visits, Hospitalizations, and Deaths 2002-2006. Atlanta,

GA: Centers for Disease Control and Prevention. Retrieved from: http://www.cdc.gov/tra

umaticbraininjury/tbi_ed.html

Fleming, J., Tooth, L., Hassell, M., & Burchan, W. (1999). Prediction of community integration

and vocational outcome 2-5 years after traumatic brain injury rehabilitation in Australia.

Brain Injury, 13(6), 417-431. doi:10.1080/026990599121476

Forslund, M. V., Roe, C., Arango-Lasprilla, J. C., Sigurdardottir, S. & Andelic, N. (2013).

Impact of personal and environmental factors on employment outcome two years after

moderate-to-severe traumatic brain injury. Journal of Rehabilitation Medicine, 45(8),

801-807. doi: 10.2340/16501977-1168

Fraser, R., Machamer, J., Temkin, N., Dikmen, S., & Doctor, J. (2006). Return to work in

traumatic brain injury (TBI): A perspective on capacity for job complexity. Journal of


Vocational Rehabilitation, 25(3), 141-148. Retrieved from: http://iospress.metapress.com

/content/103174/?p=87746e411d9545b9b4d4af586f6e7f98&pi=0

French, L. M., Anderson-Barnes, V., Ryan, L. M., Zazeckis, T. M., & Harvey, S. (2012).

Neuropsychological Practice in the Military. In C. H. Kennedy & E. A. Zillmer (Eds.)

Military Psychology: Clinical and Operational Applications (2nd ed.) New York: The

Guilford Press.

Frencham, K. A. R., Fox, A. M., & Maybery, M. T. (2005). Neuropsychological studies of mild

traumatic brain injury: A meta-analytic review of research since 1995. Journal of Clinical

and Experimental Neuropsychology, 27(3), 334-351. doi: 10.1080/13803390490520328

Gary, K. W., Arango-Lasprilla, J. C., Ketchum, J. M., Kreutzer, J. S., Copolillo, A., Novack, T.

A., & Jha, A. (2009). Racial differences in employment outcome after traumatic brain

injury at 1, 2, and 5 years postinjury. Archives of Physical Medicine and Rehabilitation,

90(10), 1699-1707. doi: doi:10.1016/j.apmr.2009.04.014

Geurtsen, G. J., van Heugten, C. M., Martina, J. D., & Geurts, A. C. H. (2010). Comprehensive

Rehabilitation Programmes in the Chronic Phase after Severe Brain Injury: A Systematic

Review. Journal of Rehabilitation Medicine, 42(2), 97-110. doi: 10.2340/16501977-0508

Grauwmeijer, E., Heijenbrok-Kal, M. H., Haitsma, I. K., & Ribbers, G. M. (2012). A prospective

Study on Employment Outcome 3 Years After Moderate to Severe Traumatic Brain

Injury. Archives of Physical Medicine and Rehabilitation, 93(6), 993-999. doi:

10.1016/j.apmr.2012.01.018

Guerin, F., Kennepohl, S., Leveille, G., Dominique, A., & McKerral, M. (2006). Vocational

outcome indicators in atypically recovering mild TBI: A post-intervention study.


Neurorehabilitation, 21(4), 295-303. Retrieved from: http://iospress.metapress.com/conte

nt/103177/?p=e63a4a67a0a74d779d92138ecaefd237&pi=0

Haefke, C., Sonntag, M., & van Rens, T. (2013). Wage rigidity and job creation. Journal of

Monetary Economics, 60(8), 887-899. doi: 10.1016/j.jmoneco.2013.09.003

Han, S. D., Suzuki, H., Drake, A. I., Jak, A. J., Houston, W. S., & Bondi, M. W. (2009). Clinical,

cognitive, and genetic predictors of change in job status following traumatic brain injury

in a military population. Journal of Head Trauma Rehabilitation, 24(1), 57-64. doi:

10.1097/HTR.0b013e3181957055

Hanlon, R. E., Demery, J. A., Martinovich, Z., & Kelly, J. P. (1999). Effects of acute injury

characteristics on neuropsychological status and vocational outcome following mild

traumatic brain injury. Brain Injury, 13(11), 873-887. doi:10.1080/026990599121070

Harris, I., Mulford, J., Solomon, M., van Gelder, J. M., & Young, J. (2005). Association

Between Compensation Status and Outcome After Surgery. JAMA, 293(13), 1644-1652.

doi: 10.1001/jama.293.13.1644

Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. L., & Millis, S. R. (2009). American

Academy of Clinical Neuropsychology consensus conference statement on the

neuropsychological assessment of effort, response bias, and malingering. The Clinical

Neuropsychologist, 23(7), 1093-1129. doi: 10.1080/13854040903155063

Henry, J. D., & Crawford, J. R. (2004). A meta-analytic review of verbal fluency performance

following focal cortical lesions. Neuropsychology, 18(2), 284-295. doi: 10.1037/0894-

4105.18.2.284


Hoofien, D., Barak, O., Vakil, E., & Gilboa, A. (2005). Symptom Checklist-90 Revised scores in

persons with traumatic brain injury: Affective reactions or neurobehavioral outcomes of

the injury? Applied Neuropsychology, 12(1), 30-39. doi: 10.1207/s15324826an1201_6

Iverson, G. L. (2001). Interpreting change on the WAIS-III/WMS-III in clinical samples.

Archives of Clinical Neuropsychology, 16(2), 183-191. doi: 10.1016/S0887-

6177(00)00060-3

Iverson, G. L. (2005). Outcome from mild traumatic brain injury. Current Opinion in Psychiatry,

18(3), 301-317.

Iverson, G. L. (2010). Mild traumatic brain injury meta-analyses can obscure individual

differences. Brain Injury, 24(10), 1246-1255. doi: 10.3109/02699052.2010.490513

Ivnik, R. J., Sharbrough, F. W., & Laws, E. R. (1988). Anterior temporal lobectomy for the

control of partial complex seizures: Information for counseling patients. Mayo Clinic

Proceedings, 63(8), 783-793.

Johansson, U., & Bernspang, B. (2001). Predicting return to work after brain injury using

occupational therapy assessments. Disability and Rehabilitation, 23(11), 474-480. doi:

10.1080/09638280010010688

Johnstone, B., Vieth, A. Z., Johnson, J. C., & Shaw, J. A. (2000). Recall as a function of single

versus multiple trials: Implications for rehabilitation. Rehabilitation Psychology, 45(1), 3-

19. doi: 10.1037/0090-5550.45.1.3

Jourdan, C., Bosserelle, V., Azerad, S., Ghout, I., Bayen, E., Aegerter, P,…Azouvi, P. (2013).

Predictive factors for the 1-year outcome of a cohort of patients with severe traumatic

brain injury (TBI): Results from the PariS-TBI study. Brain Injury, 27(9), 1000-1007.

doi: 10.3109/02699052.2013.794971


Kaplan, E., Goodglass, H., & Weintraub, S. (1983). Boston Naming Test. Philadelphia: Lea &

Febiger.

Kay, T., Harrington, D. E., Adams, R., Anderson, T., Berrol, S., Cicerone, K,…Malec, J. (1993).

Definition of mild traumatic brain injury. Journal of Head Trauma Rehabilitation, 8(3),

86-87.

Kennedy, J. E., Clement, P. F., & Curtis, G. (2003). WAIS-III Processing Speed Index scores

after TBI: The influence of working memory, psychomotor speed and perceptual

processing. The Clinical Neuropsychologist, 17(3), 303-307. doi:

10.1076/clin.17.3.303.18091

Ketchum, J. M., Getachew, M. A., Krch, D., Banos, J. H., Kolakowsky-Hayner, S. A., Lequerica,

A.,…Arango-Lasprilla, J. C. (2012). Early predictors of employment outcomes 1 year

post traumatic brain injury in a population of Hispanic individuals. NeuroRehabilitation,

30(1), 13-22. doi: 10.3233/NRE-2011-0723

Keyser-Marcus L. A., Bricout, J. C., Wehman, P., Campbell, L. R., Cifu, D. X., Englander, J.,…

Zafonte, R. D. (2002). Acute predictors of return to employment after traumatic brain

injury: A longitudinal follow-up. Archives of Physical Medicine and Rehabilitation,

83(5), 635-641. doi:10.1053/apmr.2002.31605

King, N. S., Crawford, S., Wenden, F.J., Moss, N.E.G., Wade, D.T. (1995). The Rivermead Post

Concussion Symptoms Questionnaire: a measure of symptoms commonly experienced

after head injury and its reliability. Journal of Neurology, 242, 587-592.

Klove, H. (1963). Clinical Neuropsychology. In F. M. Forster (Ed.), The medical clinics of North

America. New York: Saunders.


Konrad, C., Geburek, A. J., Rist, F., Blumenroth, H., Fischer, B., Husstedt, I.,…Lohmann, H.

(2011). Long-term cognitive and emotional consequences of mild traumatic brain injury.

Psychological Medicine, 41(6), 1197-1211. doi: 10.1017/S0033291710001728

Kozel, J. J., & Meyers, J. E. (1998). A cross-validation study of the Victoria revision of the

Category Test. Archives of Clinical Neuropsychology, 13(3), 327-332. doi:

10.1016/S0887-6177(97)00036-X

Kreutzer, J. S., Marwitz, J. H., Walker, W., Sander, A., Sherer, M., Bogner, J.,…Bushnik, T.

(2003). Moderating factors in return to work and job stability after traumatic brain injury.

The Journal of Head Trauma Rehabilitation, 18(2), 128-138. Retrieved from: http://journ

als.lww.com/headtraumarehab/pages/default.aspx

Krupp, L. B., LaRocca, N. G., Muir-Nash, J., & Steinberg, A. D. (1989). The fatigue severity

scale. Application to patients with multiple sclerosis and systemic lupus erythematosus.

Archives of Neurology, 46(10), 1121-1123. Retrieved from: http://archneur.ama-assn.org/

Labbe, D. R., Vance, D. E., Wadley, V., & Novack, T. A. (2014). Predictors of driving

avoidance and exposure following traumatic brain injury. The Journal of Head Trauma

Rehabilitation, 29(2), 185-192. doi: 10.1097/HTR.0b013e3182795211

Langeluddecke, P. M., & Lucas, S. K. (2003). Wechsler Adult Intelligence Scale – Third Edition

findings in relation to severity of brain injury in litigants. The Clinical

Neuropsychologist, 17(2), 273-284. doi:10.1076/clin.17.2.273.16499

Leeflang, M. M., Moons, K. G., Reitsma, J. B., & Zwinderman, A.H. (2008). Bias in sensitivity

and specificity caused by data-driven selection of optimal cutoff values: mechanisms,

magnitude, and solutions. Clinical Chemistry, 54(4), 729-737. doi:

10.1373/clinchem.2007.096032


Lemay, S., Bedard, M. A., Rouleau, I., & Tremblay, P. L. G. (2004). Practice effect and test-

retest reliability of attentional and executive tests in middle-aged to elderly participants.

The Clinical Neuropsychologist, 18(2), 284-302. doi: 10.1080/13854040490501718

Levi, Y., Rassovsky, Y., Agranov, E., Sela-Kaufman, M., & Vakil, E. (2013). Cognitive Reserve

Components as Expressed in Traumatic Brain Injury. Journal of the International

Neuropsychological Society, 19(6), 664-671. doi: 10.1017/S1355617713000192

Levin H. S., High, W., Goethe, K., Sisson, R., Overall, J., Rhoades, H.,…Gary, H. E. (1987).

The neurobehavioral rating scale: Assessment of the behavioral sequelae of head injury

by the clinician. Journal of Neurology, Neurosurgery, and Psychiatry, 50(2), 183–193.

doi: 10.1136/jnnp.50.2.183

Lezak, M. D., Howieson, D. B., & Loring, D. W. (2004). Neuropsychological Assessment (4th

ed.). New York: Oxford University Press.

Lopez, M. N., Charter, R. A., & Newman, J. R. (2000). Psychometric properties of the Halstead

Category Test. The Clinical Neuropsychologist, 14(2), 157-161. doi: 10.1076/1385-

4046(200005)14:2;1-Z;FT157

Lundqvist, A. & Samuelsson, K. (2012). Return to work after acquired brain injury: A patient

perspective. Brain Injury, 26(13-14), 1574-1585. doi: 10.3109/02699052.2012.698363

Machamer, J., Temkin, N., Fraser, R., Doctor, J. N., & Dikmen, S. (2005). Stability of

employment after traumatic brain injury. Journal of the International Neuropsychological

Society, 11(7), 807-816. doi: 10.1017/S135561770505099X

MacMillan, P. J., Hart, R. P., Martelli, M. F., & Zasler, N. D. (2002). Pre-injury adaptation

following traumatic brain injury. Brain Injury, 16(1), 41-49. doi:10.1080/0269905011008

812


Magalhaes, S. S., Malloy-Diniz, L. F., & Hamdan, A. C. (2012). Validity Convergent and

Reliability Test-retest of the Rey Auditory Verbal Learning Test. Clinical

Neuropsychiatry, 9(3), 129-137.

McCrea, M. (2008). Mild Traumatic Brain Injury and Postconcussion Syndrome: The New

Evidence Base for Diagnosis and Treatment. New York: Oxford University Press.

McKay, C., Casey, J. E., Wertheimer, J., & Fichtenberg, N. L. (2007). Reliability and validity of

the RBANS in a traumatic brain-injured sample. Archives of Clinical Neuropsychology,

22(1), 91-98. doi: 10.1016/j.acn.2006.11.00

Mertens, V. B., Gagnon, M., Coulombe, D., & Messier, C. (2006). Exploratory factor analysis of

neuropsychological tests and their relationship to the Brown-Peterson task. Archives of

Clinical Neuropsychology, 21(7), 733-739. doi: 10.1016/j.acn.2006.08.00

Meyers, J., Reinsch-Boothby, L., & Miller, R. (2011). Does the Source of a Forensic Referral

Affect Neuropsychological Test Performance on a Standardized Battery of Tests? The

Clinical Neuropsychologist, 25(3), 477-487. doi: 10.1080/13854046.2011.554442

Meyers, J. E., & Meyers, K. R. (1995). Rey Complex Figure Test and Recognition Trial:

Professional Manual. Odessa, FL: Psychological Assessment Resources, Inc.

Meyers, J. E., & Rohling, M. L. (2004). Validation of the Meyers Short Battery on mild TBI

patients. Archives of Clinical Neuropsychology, 19(5), 637-651. doi: 10.1016/j.acn.2003.

08.007

Meyers, J. E., & Rohling, M. L. (2009). CT and MRI correlations with neuropsychological tests.

Applied Neuropsychology, 16(4), 237-253. doi: 10.1080/09084280903098752


Meyers, J. E., & Volbrecht, M. E. (2003). A validation of multiple malingering detection

methods in a large clinical sample. Archives of Clinical Neuropsychology, 18(3), 261-

276. doi: 10.1016/S0887-6177(02)00136-1

Meyers, J. E., Volbrecht, M., Axelrod, B. N., & Reinsch-Boothby, L. (2011). Embedded

Symptom Validity Tests and Overall Neuropsychological Test Performance. Archives of

Clinical Neuropsychology, 26(1), 8-15. doi: 10.1093/arclin/acq083

Meyers, J. E., Volbrecht, M., & Kaster-Bundgaard, J. (1999). Driving is more than pedal

pushing. Applied Neuropsychology, 6(3), 154-164. doi: 10.1207/s15324826an0603_3

Miceli, G., Caltagirone, C., Gainotti, G., Masullo, C., & Silveri, M. C. (1981).

Neuropsychological correlates of localized cerebral lesions in non-aphasic brain-damaged

patients. Journal of Clinical Neuropsychology, 3(1), 53-63. doi:

10.1080/01688638108403113

Miller, L. S., & Rohling, M. L. (2001). A statistical interpretive method for neuropsychological

data. Neuropsychology Review, 11(3), 143-169. doi: 10.1023/A:1016602708066

Mitrushina, M., Boone, K. B., Razani, J., & D’Elia, L. F. (2005). Handbook of Normative Data

for Neuropsychological Assessment. New York: Oxford University Press.

Mitrushina, M., & Satz, P. (1991). Effect of repeated administration of a neuropsychological

battery in the elderly. Journal of Clinical Psychology, 47(6), 790-801. doi: 10.1002/1097-

4679(199111)47:6<790::AID-JCLP2270470610>3.0.CO;2-C

Moscarini, G., & Postel-Vinay, F. (2012). The Contribution of Large and Small Employers to

Job Creation in Times of High and Low Unemployment. The American Economic

Review, 102(6), 2509-2539. doi: http://dx.doi.org/10.1257/aer.102.6.2509


Muller, H., Hasse-Sander, I., Horn, R., Helmstaedter, C., & Elger, C. E. (1997). Rey Auditory-

Verbal Learning Test: Structure of a Modified German Version. Journal of Clinical

Psychology, 53(7), 663-671. doi: 10.1002/(SICI)1097-4679(199711)53:7<663::AID-

JCLP4>3.0.CO;2-J

Nakase-Richardson, R., Sherer, M., Barnett, S. D., Yablon, S. A., Evans, C. C., Kretzmer, T,…

Modarres, M. (2013). Prospective Evaluation of the Nature, Course, and Impact of Acute

Sleep Abnormality After Traumatic Brain Injury. Archives of Physical Medicine and

Rehabilitation, 94(5), 875-882. doi: 10.1016/j.apmr.2013.01.001

Norris, M. P., Blankenship-Reuter, L., Snow-Turek, A. L., & Finch, J. (1995). Influence of

depression on verbal fluency performance. Aging and Cognition, 2(3), 206-215. doi:

10.1080/13825589508256598

O’Donnell, J. P., McGregor, L. A., Dabrowski, J. J., Oestreicher, J. M., & Romero, J. J. (1994).

Construct validity of neuropsychological tests of conceptual and attentional abilities.

Journal of Clinical Psychology, 50(4), 596-600. doi: 10.1002/1097-4679(199407)50:4<5

96::AID-JCLP2270500416>3.0.CO;2-S

Ostrosky-Solis, F., Jaime, R. M., & Ardila, A. (1998). Memory abilities during normal aging.

International Journal of Neuroscience, 93(1-2), 151-162.

Ownsworth, T., & McKenna, K. (2004). Investigation of factors related to employment outcome

following traumatic brain injury: A critical review and conceptual model. Disability and

Rehabilitation, 26(13), 765-784. doi:10.1080/09638280410001696700

Parkin, A. J. (1998). The central executive does not exist. Journal of the International

Neuropsychological Society, 4(5), 518-522. Retrieved from: http://www2.psych.ubc.ca/~

pgraf/Psy583Readings/Parkin%201998.pdf


Parks, J. K., Diaz-Arrastia, R., Gentilello, L. M., & Shafi, S. (2010). Postinjury employment as a

surrogate for functional outcomes: A quality indicator for trauma systems. Proceedings

(Baylor University Medical Center), 23(4), 355-358. Retrieved from: http://www.baylorh

ealth.edu/Research/Proceedings/Pages/default.aspx

Pertab, J. L., James, K. M., and Bigler, E. D. (2009). Limitations of mild traumatic brain injury

meta-analyses. Brain Injury, 23(6), 498-508. doi: 10.1080/02699050902927984

Pilgrim, B. M., Meyers, J. E., Bayless, J., & Whetstone, M. M. (1999). Validity of the Ward

seven-subtest WAIS-III short form in a neuropsychological population. Applied

Neuropsychology, 6(4), 243-246. doi: 10.1207/s15324826an0604_7

Ponsford, J., Downing, M., Olver, J., Ponsford, M., Acher, R., Carty, M., & Spitz, G. (2014).

Longitudinal follow-up of patients with traumatic brain injury: Outcome at 2, 5, and 10-

years post-injury. Journal of Neurotrauma, 31(1), 64-77. doi: 10.1089/neu.2013.2997

Ponton, M. O., Gonzalez, J. J., Hernandez, I., Herrera, L., & Higareda, I. (2000). Factor analysis

of the Neuropsychological Screening Battery for Hispanics (NeSBHIS). Applied

Neuropsychology, 7(1), 32-39. doi: 10.1207/S15324826AN0701_5

Prigatano, G. P. (1999). Impaired awareness, finger tapping, and rehabilitation outcome after

brain injury. Rehabilitation Psychology, 44(2), 145-159. doi: 10.1037/0090-

5550.44.2.145

Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical

neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA

Division 40 members. Archives of Clinical Neuropsychology, 20(1), 33-65. doi:

10.1016/j.acn.2004.02.005


Rakic, T., Steffens, M. C., Mummendey, A. (2011). When it matters how you pronounce it: The

influence of regional accents on job interview outcome. British Journal of Psychology,

102(4), 868-883. doi: 10.1111/j.2044-8295.2011.02051.x

Randolph, C. (1998). RBANS manual: Repeatable Battery for the Assessment of

Neuropsychological Status. San Antonio, TX: The Psychological Corporation.

Rappaport, M., Hall, K. M., Hopkins, K., Belleza, T., & Cope, D. N. (1982). Disability rating

scale for severe head trauma: Coma to community. Archives of Physical Medicine and

Rehabilitation, 63(3), 118-123.

Rapport, L. J., Hanks, R. A., & Bryer, R. C. (2006). Barriers to Driving and Community

Integration After Traumatic Brain Injury. Journal of Head Trauma Rehabilitation, 21(1),

34-44.

Rapport, L. J., Bryer, R. C., & Hanks, R. A. (2008). Driving and Community Integration After

Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 89(5), 922-

930. doi: 10.1016/j.apmr.2008.01.009

Raven, J., Raven, J. C., & Court, J. H. (1993). Manual for the Raven’s Progressive Matrices and

Vocabulary Scales. Oxford, England: Oxford Psychologists Press.

Reid-Arndt, S. A., Nehl, C., & Hinkebein, J. (2007). The Frontal Systems Behaviour Scale

(FrSBe) as a predictor of community integration following a traumatic brain injury. Brain

Injury, 21(13-14), 1361-1369. doi: 10.1080/02699050701785062

Reitan, R. M. (1958). Validity of the Trail Making Test as an indicator of organic brain damage.

Perceptual and Motor Skills, 8(3), 271-276. doi: 10.2466/pms.1958.8.3.271

Reitan, R. M., & Wolfson, D. (1993). The Halstead-Reitan Neuropsychological Test Battery:

Theory and clinical applications (2nd ed.). Tucson, AZ: Neuropsychology Press.


Rey, A. (1964). L’examen clinique en psychologie. Paris: Presses Universitaires de France.

Roberts, M. A., Persinger, M. A., Grote, C., Evertowski, L. M., Springer, J. A., Tuten, T.,…

Baglio, C.S. (1994). The Dichotic Word Listening Test: preliminary observations in

American and Canadian samples. Applied Neuropsychology, 1(1-2), 45-56.

Robertson, I. (2008). Traumatic brain injury: Recovery, prediction, and the clinician. Archives of

Physical Medicine and Rehabilitation, 89(12 Suppl. 2), 1-2. doi:10.1016/j.apmr.2008.10.

001

Rohling, M. L., Binder, L. M., Demakis, G. J., Larrabee, G. J., Ploetz, D. M., & Langhinrichsen-

Rohling, J. (2011). A Meta-Analysis of Neuropsychological Outcome After Mild

Traumatic Brain Injury: Re-analyses and Reconsiderations of Binder et al. (1997),

Frencham et al. (2005), and Pertab et al. (2009). The Clinical Neuropsychologist, 25(4),

608-623. doi: 10.1080/13854046.2011.565076

Rohling, M. L., Meyers, J. E., & Millis, S. R. (2003). Neuropsychological impairment following

traumatic brain injury: A dose-response analysis. The Clinical Neuropsychologist, 17(3),

289-302. doi: 10.1076/clin.17.3.289.18086

Ross, S. R., Millis, S. R., & Rosenthal, M. (1997). Neuropsychological prediction of

psychosocial outcome after traumatic brain injury. Applied Neuropsychology, 4(3), 165-

170. doi: 10.1207/s15324826an0403_4

Ross, T. P., Furr, A. E., Carter, S. E., & Weinberg, M. (2006). The psychometric equivalence of

two alternate forms of the Controlled Oral Word Association Test. The Clinical

Neuropsychologist, 20(3), 414-431. doi: 10.1080/13854040590967153

Royan, J., Tombaugh, T. N., Rees, L., & Francis, M. (2004). The Adjusting-Paced Serial

Addition Test (Adjusting-PASAT): Thresholds for speed of information processing as a


function of stimulus modality and problem complexity. Archives of Clinical

Neuropsychology, 19(1), 131-143. doi: 10.1016/S0887-6177(02)00216-0

Ruff, R. M., Light, R. H., Parker, S. B., & Levin, H. S. (1996). Benton Controlled Oral Word

Association Test: Reliability and updated norms. Archives of Clinical Neuropsychology,

11(4), 329-338. doi: 10.1016/0887-6177(95)00033-X

Ryan, J. J., Rosenberg, S. J., & Mittenberg, W. (1984). Factor analysis of the Rey Auditory-

Verbal Learning Test. The International Journal of Clinical Neuropsychology, 6(4), 239-

241.

Ryu, W. H. A., Cullen, N. K., & Bayley, M. T. (2010). Early neuropsychological tests as

correlates of productivity 1 year after traumatic brain injury: A preliminary matched case-

control study. International Journal of Rehabilitation Research, 33(1), 84-87. doi:

10.1097/MRR.0b013e32832e6b4b

Salgado, J. F., & Moscoso, S. (2002). Comprehensive meta-analysis of the construct validity of

the employment interview. European Journal of Work and Organizational Psychology,

11(3), 299-324. doi: 10.1080/13594320244000184

Salthouse, T. A. (2005). Relations between cognitive abilities and measures of executive

functioning. Neuropsychology, 19(4), 532-545. doi: 10.1037/0894-4105.19.4.532

Schneider, E. B., Sur, S., Raymont, V., Duckworth, J., Kowalski, R. G., Efron, D. T.,…Stevens,

R. D. (2014). Functional recovery after moderate/severe traumatic brain injury: A role for

cognitive reserve? Neurology, 82(18), 1636-1642. doi:

10.1212/WNL.0000000000000379


Schneider, J. C., Bassi, S., & Ryan, C. M. (2011). Employment Outcomes After Burn Injury: A

Comparison of Those Burned at Work and Those Burned Outside of Work. Journal of

Burn Care & Research, 32(2), 294-301. doi: 10.1097/BCR.0b013e31820aaf56

Schonberger, M., Ponsford, J., Olver, J., Ponsford, M., & Wirtz, M. (2011). Prediction of

functional and employment outcome 1 year after traumatic brain injury: a structural

equation modeling approach. Journal of Neurology, Neurosurgery, and Psychiatry, 82(8),

936-941. doi: 10.1136/jnnp.2010.210021

Shah, S., Vanclay, F., & Cooper, B. (1989). Improving the sensitivity of the Barthel Index for

stroke rehabilitation. Journal of Clinical Epidemiology, 42(8), 703-709. doi:

10.1016/0895-4356(89)90065-6

Shames, J., Treger, I., Ring, H., & Giaquinto, S. (2007). Return to work following traumatic

brain injury: Trends and challenges. Disability and Rehabilitation, 29(17), 1387-1395.

doi:10.1080/09638280701315011

Sherer, M., Novack, T. A., Sander, A. M., Struchen, M. A., Alderson, A., & Thompson, R. N.

(2002). Neuropsychological assessment and employment outcome after traumatic brain

injury: A review. The Clinical Neuropsychologist, 16(2), 157-178. doi: 10.1076/clin.16.2.

157.13238

Sherer, M., Sander, A. M., Nick, T. G., High, W. M., Malec, J. F., Rosenthal, M. (2002). Early

Cognitive Status and Productivity Outcome After Traumatic Brain Injury: Findings From

the TBI Model Systems. Archives of Physical Medicine and Rehabilitation, 83(2), 183-

192. doi: http://dx.doi.org/10.1053/apmr.2002.28802


Sherman, E. M. S., Strauss, E., Spellacy, F., & Hunter, M. (1995). Construct validity of WAIS-R

factors: Neuropsychological test correlates in adults referred for evaluation of possible

head injury. Psychological Assessment, 7(4), 440-444.

Sherrill, R. E. (1987). Options for shortening Halstead’s category test for adults. Archives of

Clinical Neuropsychology, 2(4), 343-352. doi: 10.1016/0887-6177(87)90003-5

Shimer, R. (2012). Reassessing the ins and outs of unemployment. Review of Economic

Dynamics, 15, 127-148. doi: 10.1016/j.red.2012.02.001

Sigurdardottir, S., Andelic, N., Roe, C., & Schanke, A. K. (2009). Cognitive recovery and

predictors of functional outcome 1 year after traumatic brain injury. Journal of the

International Neuropsychological Society, 15(5), 740-750. doi: 10.1017/S135561770999

0452

Sigurdardottir, S., Andelic, N., Roe, C., & Schanke, A. K. (2013). Depressive Symptoms and

Psychological Distress During the First Five Years After Traumatic Brain Injury:

Relationship With Psychological Stressors, Fatigue, and Pain. Journal of Rehabilitation

Medicine, 45(8), 808-814. doi: 10.2340/16501977-1156

Simpson, A., & Schmitter-Edgecombe, M. (2002). Prediction of employment status following

traumatic brain injury using a behavioral measure of frontal lobe functioning. Brain

Injury, 16(12), 1075-1091. doi:10.1080/02699050210155249

Soper, D. S. (Accessed January 30, 2012). The Free Statistics Calculators Website. Retrieved

from: http://www.danielsoper.com/statcalc3/

Spencer, R. J., Wendell, C. R., Giggey, P. P., Seliger, S. L., Katzel, L. I., & Waldstein, S. R.

(2013). Judgment of Line Orientation: An examination of eight short forms. Journal of


Clinical and Experimental Neuropsychology, 35(2), 160-166. doi:

10.1080/13803395.2012.760535

Spitz, G., Ponsford, J. L., Rudzki, D., Maller, J. J. (2012). Association between cognitive

performance and functional outcome following traumatic brain injury: a longitudinal

multilevel examination. Neuropsychology, 26(5), 604-612. doi: 10.1037/a0029239

Spreen, O., & Strauss, E. (1991). A Compendium of Neuropsychological Tests: Administration,

Norms and Commentary. New York: Oxford University Press.

Spreen, O., & Strauss, E. (1998). A Compendium of Neuropsychological Tests: Administration,

Norms, and Commentary (2nd ed.). New York: Oxford University Press.

Stallings, G., Boake, C., & Sherer, M. (1995). Comparison of the California Verbal Learning

Test and the Rey Auditory-Verbal Learning Test in head-injured patients. Journal of

Clinical and Experimental Neuropsychology, 17(5), 706-712. doi: 10.1080/01688639508

405160

Strauss, E., Sherman, E. M. S., & Spreen, O. (2006). A Compendium of Neuropsychological

Tests: Administration, Norms, and Commentary (3rd ed.). New York: Oxford University

Press.

Stern, Y. (2002). What is cognitive reserve? Theory and research application of the reserve

concept. Journal of the International Neuropsychological Society, 8(3), 448-460. doi:

10.1017.S1355617701020240

Suhr, J. A., & Gunstad, J. (2000). The effects of coaching on the sensitivity and specificity of

malingering measures. Archives of Clinical Neuropsychology, 15(5), 415-424. doi:

10.1016/S0887-6177(99)00033-5


Tabachnick, B. G., & Fidell, L. S. (2007). Using Multivariate Statistics (5th ed.). Boston, MA:

Pearson.

Taylor, J. E., Leung, J., & Deane, F. P. (2011). Neuropsychological assessment of fitness to drive

following acquired cognitive impairment. Brain Injury, 25(5), 471-487. doi:

10.3109/02699052.2011.559609

The Management of Concussion/mTBI Working Group (2009). VA/DoD Clinical Practice

Guideline for Management of Concussion/Mild Traumatic Brain Injury. Journal of

Rehabilitation Research and Development, 46(6), CP1-68.

The Psychological Corporation (2002). WAIS-III/WMS-III: Updated Technical Manual. San

Antonio: The Psychological Corporation.

The Traumatic Brain Injury Model Systems National Data and Statistical Center (2010). TBI

Model Systems Brochure [Brochure]. Retrieved from https://www.tbindsc.org/Documents

/TBIModel%20SystemsBrochure2010.pdf

Tiffin, J. (1968). Purdue Pegboard Examiner Manual. Chicago: Science Research Associates,

Inc.

Tombaugh, T. N., Kozak, J., & Rees, L. (1999). Normative data stratified by age and education

for two measures of verbal fluency: FAS and animal naming. Archives of Clinical

Neuropsychology, 14(2), 167-177. doi: 10.1093/arclin/14.2.167

Treccani, B., & Cubelli, R. (2011). The need for a revised version of the Benton judgment of line

orientation test. Journal of Clinical and Experimental Neuropsychology, 33(2), 249-256.

doi: 10.1080/13803395.2010.511150


Triggs, W. J., Calvanio, R., Levine, M., Heaton, R. K., & Heilman, K. M. (2000). Predicting

hand preference with performance on motor tasks. Cortex, 36(5), 679-689. doi:

10.1016/S0010-9452(08)70545-8

Usui, N., Haji, T., Maruyama, M., Katsuyama, N., Uchida, S., Hozawa, A,…Taira, M. (2009).

Cortical areas related to performance of WAIS Digit Symbol Test: A functional imaging

study. Neuroscience Letters, 463(1), 1-5. doi: 10.1016/j.neulet.2009.07.048

Vakil, E. (2005). The effect of moderate to severe traumatic brain injury (TBI) on different

aspects of memory: A selective review. Journal of Clinical and Experimental

Neuropsychology, 27(8), 977-1021. doi: 10.1080/13803390490919245

Vakil, E., & Blachstein, H. (1993). Rey Auditory Verbal Learning Test: Structure analysis.

Journal of Clinical Psychology, 49(6), 883-890. doi: 10.1002/1097-

4679(199311)49:6<883::AID-JCLP2270490616>3.0.CO;2-6

van den Burg, W., & Kingma, A. (1999). Performance of 225 Dutch school children on Rey’s

Auditory Verbal Learning Test (AVLT): Parallel test-retest reliabilities with an interval

of 3 months and normative data. Archives of Clinical Neuropsychology, 14(6), 545-559.

doi: 10.1016/S0887-6177(98)00042-0

van der Heijden, P., & Donders, J. (2003). A confirmatory factor analysis of the WAIS-III in

patients with traumatic brain injury. Journal of Clinical and Experimental

Neuropsychology, 25(1), 59-65. doi: 10.1076/jcen.25.1.59.13627

van der Horn, H., Spikman, J. M., Jacobs, B., van der Naalt, J. (2013). Postconcussive

Complaints, Anxiety, and Depression Related to Vocational Outcome in Minor to Severe

Traumatic Brain Injury. Archives of Physical Medicine and Rehabilitation, 94(5), 867-

874. doi: 10.1016/j.apmr.2012.11.039


Varney, N. R., & Stewart, H. (2004). Is impaired executive function a single or multidimentional

disability? Applied Neuropsychology, 11(4), 227-232. doi: 10.1207/s15324826an1104_10

Volbrecht, M. E., Meyers, J. E., & Kaster-Bundgaard, J. (2000). Neuropsychological outcome of

head injury using a short battery. Archives of Clinical Neuropsychology, 15(3), 251-265.

doi: doi:10.1016/S0887-6177(99)00016-5

Walker, W. C., Marwitz, J. H., Kreutzer, J. S., Hart, T., & Novack, T. A. (2006). Occupational

categories and return to work following traumatic brain injury: A multicenter study.

Archives of Physical Medicine and Rehabilitation, 87(12), 1576-1582. doi:10.1016/j.apm

r.2006.08.335

Ward, L. (1990). Prediction of verbal, performance and full scale IQs from seven subtests of the

WAIS-R. Journal of Clinical Psychology, 46(4), 436–440. doi: 10.1002/1097-

4679(199007)46:4<436::AID-JCLP2270460411>3.0.CO;2-M

Ward, L. C., Ryan, J. J., & Axelrod, B. N. (2000). Confirmatory factor analyses of the WAIS-III

standardization data. Psychological Assessment, 12(3), 341-345. doi: 10.1037/1040-

3590.12.3.341

Wechsler, D. (1987). Wechsler Memory Scale—Revised. New York: The Psychological

Corporation.

Wechsler, D. (1997). Wechsler Adult Intelligence Scale-III. San Antonio, TX: The Psychological

Corporation.

Wechsler, D. A. (1945). Standardized memory scale for clinical use. Journal of Psychology,

19(1), 87-95.

Wehman, P., Targett, P., West, M., & Kregel, J. (2005). Productive work and employment for

persons with traumatic brain injury: What have we learned after 20 years? Journal of


Head Trauma Rehabilitation, 20(2), 115-127. Retrieved from: http://journals.lww.com/he

adtraumarehab/pages/default.aspx

Wetter, M. W., & Corrigan, S. K. (1995). Providing information to clients about psychological

tests: A survey of attorneys’ and law students’ attitudes. Professional Psychology:

Research and Practice, 26(5), 474-477. doi: 10.1037/0735-7028.26.5.474

Whelan-Goodinson, R., Ponsford, J., Johnston, L., & Grant, F. (2009). Psychiatric disorders

following traumatic brain injury: Their nature and frequency. Journal of Head Trauma

Rehabilitation, 24(5), 324-332. doi: 10.1097/HTR.0b013e3181a712aa

Wilde, M. C., Boake, C., & Sherer, M. (2000). Wechsler Adult Intelligence Scale – Revised

block design broken configuration errors in nonpenetrating Traumatic Brain Injury.

Applied Neuropsychology, 7(4), 208-214. doi: 10.1207/S15324826AN0704_2

Williams, M. W., Rapport, L. J., Hanks, R. A., Millis, S. R., & Greene, H. A. (2013).

Incremental Validity of Neuropsycholgoical Evaluations to Computed Tomography in

Predicting Long-Term Outcomes after Traumatic Brain Injury. The Clinical

Neuropsychologist, 27(3), 356-375. doi: 10.1080/13854046.2013.765507

Wilson, B., Cockburn, J., Baddeley, A., & Hiorns, R. (1988). The development and validation of

a battery for detecting and monitoring everyday memory problems. Journal of Clinical

and Experimental Neuropsychology, 11(6), 855-870, doi: 10.1080/01688638908400940

Yasuda, S., Wehman, P., Targett, P., Cifu, D., & West, M. (2001). Return to work for persons

with traumatic brain injury. American Journal of Physical Medicine and Rehabilitation,

80(11), 852-864. Retrieved from: http://journals.lww.com/ajpmr/pages/default.aspx


Yates, B. T., & Taub, J. (2003). Assessing the costs, benefits, cost-effectiveness, and cost-benefit

of psychological assessment: we should, we can, and here's how. Psychological

Assessment, 15(4), 478-495. doi: 10.1037/1040-3590.15.4.478

Zakzanis, K. K., McDonald, K., Troyer, A. K. (2013). Component analysis of verbal fluency

scores in severe traumatic brain injury. Brain Injury, 27(7-8), 903-908. doi:

10.3109/02699052.2013.775505

Zappala, G., Thiebaut de Schotten, M., & Eslinger, P. J. (2012). Traumatic brain injury and the

frontal lobes: what can we gain with diffusion tensor imaging? Cortex, 48(2), 156-165.

doi: 10.1016/j.cortex.2011.06.020

Zelazo, P. D., Carter, A., Reznick, J. S., & Frye, D. (1997). Early development of executive

function: A problem-solving framework. Review of General Psychology, 1(2), 198-226.

doi: 10.1037/1089-2680.1.2.198

Zhu, J., Tulskey, D. S., Price, L., Chen, H. (2001). WAIS-III reliability data for clinical groups.

Journal of the International Neuropsychological Society, 7(7), 862 – 866.

Zillmer, E. A., Spiers, M. V., & Culbertson, W. C. (2008). Principles of Neuropsychology (2nd

ed.). Canada: Wadsworth Cengage Learning.

ScholarSpace at University of Hawaii at Manoa: Home ......Normal structural imaging Normal or...

Documents

Transcript of ScholarSpace at University of Hawaii at Manoa: Home ......Normal structural imaging Normal or...