Participation and Environmental Factors Measures For TBI Rehabilitation

Participation and Environmental Factors

Measures For TTBI Rehabilitation

Susan Magasi

Allen Heinemann

Noelle Carlozzi

1

Presentation Overview

1. Developing a measure of community participation after rehabilitation

2. Evaluating the feasibility of computer-adapted administration of a participation measure with discharged rehabilitation patients

3. Measuring the Impact of the Environment on Home, Community and Work Participation for People with Cognitive Disabilities

2

Developing a Measure of Community Participation after

Rehabilitation

Susan Magasi

3

Presentation Objectives

(1) Describe ways to assess participation

(2) Evaluate the psychometric properties of community enfranchisement items and develop an item bank that is suitable for computer adaptive testing (CAT)

(3) Explore the validity of a composite participation score and measures of community enfranchisement

4

5

International Classification of Functioning Disability and Health

Barriers

Facilitators

Body Functions&

Structures

Activities &

Participation

Environmental Factors

Functions

Structures

Capacity

Performance

Project Task Overview

1. Conduct literature review

2. Conduct focus groups with consumers, caregivers, providers, payers, policy makers

3. Develop items and rating scales to operationalize participation

4. Conduct cognitive interviews with consumers and general public

5. Revise items, rating scales

6. Pilot test participation instrument

7. Refine instrument

8. Collect population data from persons with and without disabilities


10. Collect and analyze population data from persons with and without disabilities (Behavioral Risk Factor Surveillance System)


13. Collect 3rd round of data, augmented with panel sample

14. Evaluate CPI as part of routine post-discharge follow-up assessment - underway

6

Construct Development: Stakeholder Focus Groups

•Consumers (5)–Chicago, IL–Englewood, CO

•Caregivers (6)–Chicago, IL–Englewood, CO

•Rehabilitation professionals (4)

–Chicago, IL–Englewood, CO

•Payers (1)–Insurance Research Group

•Policy Makers (2)–Federal: Washington, DC–State: Chicago, Illinois

•Total Participants–18 groups–138 participants

7

Focus Group Questions

1. What does the word “participation” mean to you? a) What does it mean “to participate”

b) Why is participation important to you?

2. What areas of everyday life are most important to you? a) If taken away, what areas would you miss most/would be the

biggest loss to you?

3. What issues or barriers most affect your participation?

4. What are your biggest supports to participation?

5. What recommendations do you have for improving participation opportunities for people with disabilities?

8

Qualitative Analytic Procedures

• Analyses and interpretation• Open coding of data by 2

researchers, including team members with disabilities

• Topical coding to identify key emerging themes

• Detailed coding of • individual data within groups• comparative analysis across

individuals• across stakeholder groups

•Codes represent • Content themes• Narrative stories & examples

• Areas of agreement and differences on specific issues

•End product• Participation item pool that reflects the themes and concerns & priorities of multiple stakeholders 9

10

Focus Group Input

Consumers “It means to make some sort of contribution in life”“Working and living”“It's important to socialize”“Just being able to do the things that you enjoy”

Care-givers “Just doing what you want to do”“Being able to go to the store, to school, being able to do all of the

things that normal people do”

Providers “It goes beyond just daily living activities”“What you want when you want with who you want”“You are seen as having something to give”

Payers “Allowed to fail, take on challenge”“Lack of information can be as isolating as any physical barrier”

Policy Makers

“Just the stuff we do and take for granted”

11

What We Learned: Participation Values

ParticipationValues

Choice & Control

Access & Opportunity

Personal & Societal

Responsibilities

Having an Impact & Supporting

Others

Social Connection, Inclusion &

Membership

Meaningful Engagement/

Being a Part of

• Hammel J, Magasi S, Heinemann AW, Whiteneck G, Bogner J, Rodriguez E. What does participation mean? An insider perspective from people with disabilities. Disability and Rehabilitation, 30:19,1445-1460.

• Magasi S, Hammel J, Heinemann AW, Whiteneck G, Bogner J. Participation: A comparative analysis of multiple rehabilitation stakeholders’ perspectives. Journal of Rehabilitation Medicine, 41, 936-944, 2009.

Community Participation IndicatorsVersion 1

• Source Material– Focus group thematic codes – Comprehensive literature review

• Content Domains– Productive roles (student, worker, and homemaker)– Social roles (spouse or partner, family member,

caregiver, caregiver to children or aging parents, relationships with adults and friends

– Community roles (volunteer, civic, political, organization, or community group member)

12

CPI Version 1: Content Organization

• Engagement (frequency, duration of activities)• Economic / productive (5 items)• Social (5)• Community (11)

• Evaluation (agreement, satisfaction)• Economic / productive (4)• Social (5)• Community (1)

• Enfranchisement• 19 items• 4-point rating scale

13

Rating Scale Analysis of Enfranchisement Items

• BRFSS addendum N = 912• Self-identified disability = 51%• Person separation reliability

= .77 (marginal)• Rating scale

• Too easy to endorse items

• Item targeting on sample• Mistargeted• Ceiling effect

• Item statistics• No misfitting items

• No evidence of differential item functioning• Disability vs. nondisability

samples

• Assessing only frequency of activities ignores appraisal of importance

• Revision Decision• Expand enfranchisement

items• Assess activity importance,

evaluation14

Conclusions: Participation enfranchisement constitutes a new, previously unmeasured aspect of participation - one that addresses subjective perceptions rather than objective performance -with items that are clearly distinct from more generalized satisfaction with participation. The 19 enfranchisement items describe aspects of participation that may prove useful in characterizing longer-term rehabilitation outcomes.

Community Participation IndicatorsVersion 2

• 20 engagement items• Frequency of activity• Importance of activity• Evaluation of activity frequency

• 48 enfranchisement items• Higher “ceiling” sought• Revised rating scale – from 4 to 5 agreement

categories

16

Enfranchisement Items

• Retained 8 thematic domains• Expanded from 19 to 48 items• Cognitive testing with consumers• New rating scale

– All the time– Frequently– Sometimes– Seldom– Almost never

20

22

Round 3 Data: Sample Characteristics

• N = 1163• Source

– Panel survey:51%

– Community sites: 31%

– Former inpatients:18%

• Median age = 53 yr (+ 17)• Women = 49%• Race

– Caucasian72%

– African-American12%

• Hispanic 7%• Self-report 72%

• Impairments– Mobility 63%– Mental 23%– Hearing

14%– Vision

13%– Learning

11%– Communication 9%

• Disability Severity

Results: Activity Patterns

• Activity Categories• Social activities• Productive activities• Low frequency activities

• Activity Importance• Yes / No

23

Social Activities

0%

20%

40%

60%

80%

100%

Spend time wfamily

Keep in touchw family

Get out &about

Keep in touchw friends

Spend time wfriends

Socialactivities

7 days

5-6 days

3-4 days

1-2 days

None

‘Productive’ Activities

0%

20%

40%

60%

80%

100%

Work Child care Homemaintenance

Householdfinances

Volunteer Learn

35+ h

20-34 h

10-19 h

5-9 h

1-4 h

None

0%

20%

40%

60%

80%

100%

Hobbies Active recreation

Religious activities

Support Groups

Entertainment Community organizations

Civic activities

5+ d

4 d

3 d

2 d

1 d

0 d

Low Frequency Activities

27

Percent of Important Activities Performed Often Enough

Mean = 55%SD = 28N = 1149

Scoring DecisionActivities and Activity Importance

• Avoid creating a “busy-ness” index• Personal preferences, opportunities,

environmental factors influence activity patterns

• Report descriptive information about activity patterns

• Evaluate utility of “percent of important activities performed often enough”

28

Rating Scale Analysis of Enfranchisement Items

Began with all 48 items Used exploratory factor analysis to identify number of

subdomains in half the sample Used confirmatory factor analysis to demonstrate

reproducibility in the second half of the sample Used rating scale analysis with subdomains to evaluate

unidimensionality and reliability

1. Involvement in life situations

2. Control over participation

29

Involvement in Life SituationsSummary Statistics

• Person Information• How well do items differentiate the people?• Very well• Reliability .87

• Item Information• How well do the people differentiate the items?• Exceptionally well• Reliability 1.00

30

Item (hardest to easiest to endorse) MeasureModel

SEInfit

MnSqI spend time doing things that improve my community

.89 .04 1.03

I have a say on decisions in my community .61 .03 1.15I am actively involved in my community .60 .03 .71I assume leadership roles in organizations .55 .03 1.34I have influence in my community .39 .03 .84I participate in a variety of activities .01 .03 .95I contribute to the well-being of my community -.11 .03 .71I regularly seek out new challenges -.11 .03 1.09I contribute to society -.18 .03 .81People see my potential -.43 .03 .97I spend time helping others -.43 .03 1.10I feel safe participating in community activities -.77 .04 1.23People count on me -1.02 .04 1.19

Involvement in Life SituationsItem Statistics (13 retained items)

Control over ParticipationSummary Statistics

• Person Information• How well do items differentiate the people?• Very well• Reliability .87

• Item Information• How well do the people differentiate the items?• Exceptionally well• Reliability 1.00

• Item Fit• I speak up for myself (Infit Mean Square=1.41)

32

Control over ParticipationItem Statistics (15 retained items)

33

Item (Hardest to easiest to endorse) MeasureModel

SEInfit

MnSq

I am able to go out and have fun .75 .04 .97I actively pursue my dreams and desires .74 .04 .89I have opportunities to make new friends .72 .04 1.19I do important things with my life .45 .04 .96I live my life fully .42 .04 .81I live my life the way that I want .36 .04 .97I participate in activities when I want .10 .04 .85I participate in activities that I choose .08 .04 1.11I do things that are important to me -.14 .04 .68I am in control of my own life -.25 .04 1.00I speak up for myself -.29 .04 1.41I have choices about the activities I do -.52 .04 .92I have control over how I spend my time -.57 .04 1.08I have the freedom to make my own decisions -.74 .04 1.04I take responsibility for my own life -1.09 .05 1.23

People with More Severe Disabilities Report Less Involvement in Life Situations

Known Groups Validity Evidence

People with More Severe Disabilities Report Less Control over Participation

Known Groups Validity Evidence

Conclusions

1. Participation as measured by activity frequency, evaluation and enfranchisement items is not a unidimensional construct• Don’t model activity frequency, importance and evaluation in a

composite measure• Individuals’ preferences and opportunities vary greatly• Personal preferences determine individual’s participation profile

2. Involvement in and control over participation are distinct constructs that can be measured reliably• Expanded set of community enfranchisement items contains 2

distinct item sets that are suitable for item banking and computer adaptive administration

3. Construct validity of involvement in and control over participation is promising

36

Feasibility of Measuring Community Participation

after Rehabilitation Discharge

Allen Heinemann


1. Evaluate the feasibility of measuring community participation after inpatient rehabilitation

2. Compare the efficacy of telephone vs. web-based data collection

3. Explore the relationship between functional status and the three aspects of community participation

4. Explore demographic, impairment, and rehabilitation variables as they relate to community participation

• Sample Recruitment• Outcomes Management Systems and Analysis staff invited

discharged adult inpatients to complete study instruments after they completed a 1-month post-discharge telephone satisfaction survey.

• Survey administration• Patients chose to complete the survey on a secure web site or

using an interactive voice response system. • Questions were administered using a computer-adaptive testing

(CAT) algorithm based on calibration of the items relating to involvement in life situations and control over participation

• Data analysis• Participation data from the CAT were matched to de-identified

inpatient data and analyzed using SPSS software.

Method

39

Results

Of the 457 patients who agreed to participate in the study, 124 (27%) completed the survey.

Of the 124 patients who completed the survey, 66% did so by phone and 34% by internet.

Completion rates were similar for both modes of completion.

Subjects who agreed to participate were younger than those who did not agree to participate.

Subject who completed the survey by phone were older than those who completed the survey by internet.

40

IVR (Phone) Internet0

50

100

150

200

250

223 11080 44

NoYes

Response Rate by Telephone and Internet

41

IVR (Phone)

Internet

0 2 4 6 8 10 12 14 16 18

11.58

16.36

Mean Days to Complete Survey (p=0.049; small effect

Time (days)

Days to Complete the Instrument

42

IVR (Phone) Internet50

52

54

56

58

60

62

64

66

68

66 56

Mean difference p<0.0005; medium effectA

ge (y

ears

)

Average Age: Phone vs. Internet

43

Results Highlights, continued

Subjects who completed the CAT by phone completed the survey sooner than those who completed the survey by the internet.

Older adults reported doing enough of their important activities than did younger adults.

Functional status as measured by the FIM was unrelated to involvement, control, or important activities performed often enough.

44

45

Conclusions

• CAT administration of participation items by phone and internet is feasible and cost-effective for collection of post-rehabilitation outcome data.

• Older subjects prefer the telephone to the internet.• The three measures of community participation are

not related to FIM scores, indicating that they are measuring a unique aspect of a patients’ experience.

• Further studies are needed to evaluate the validity and reliability of participation scores.

Acknowledgements

• Outcomes Management• Tom Snyder and OMSA Staff

• Brian Arnold

47

For More Information

Developing Environmental Factor Measures for

Persons with TBI, Stroke and SCI

Noelle Carlozzi


• Understand environmental factors and how they influence participation

• Describe efforts to develop measures of environmental factors that influence participation

• Discuss preliminary findings using these measures in TBI

ICF Environmental Factors Taxonomy

Products and Technology

Natural Environment/ Human-Made Changes to Environment

Services, Systems, and

PoliciesAttitudes

Support and Relationships

50

State-of-the-Art Measurement

• Many ways to assess environmental factors

• Recent measurement advances are not captured by the current environmental factors measures

• Future work is needed to develop “smart tests” that can assess environmental factors using state-of-the-art measurement development techniques (IRT and CAT)

51

RRTC Project to Develop Environmental Factor Measures (TBI, Stroke, & SCI)• Allen W. Heinemann, PhD

• Rehabilitation Institute of Chicago

• David Tulsky, PhD • University of Michigan

• Noelle Carlozzi, PhD • University of Michigan

• Pam Kisala, MA • University of Michigan

• Sofia Garcia, PhD• Northwestern University

• Beth Hahn, MA• Northwestern University

• Susan Magasi, PhD• Northwestern University

• Jin-Shei Lai, PhD

• Northwestern University

• David Gray, PhD • Washington University

• Holly Hollingsworth, PhD • Washington University

• Joy Hammel, PhD• University of Illinois at Chicago

RRTC Project Progress

• Year 1– Analyzed existing focus group data– Reviewed environmental factors literature and

current measures– Developed conceptual framework– Pooled, binned, winnowed, wrote items

• Year 2– Completed several rounds of cognitive testing– Completed pilot data collection– Data analysis in progress 53

54

Conceptual Framework

Economic Quality of Life

Systems Services and

Policies

Built and Natural Environment

Assistive Technology

Access to Information and

Technology

Social Environment

Environmental Factors

Binned and Winnowed Items

55

Domain Binned Winnowed Draft Items

Economic Quality of Life 42 35 44Systems, Services & Policies 411 35 40Built & Natural Environment 684 88 75

Assistive Technology178 104 24

Access to Information & Technology 112 37 32Social Supports & Attitudes 710 59 65Total 2212 358 280


Financial influence on:

• Satisfaction with income level

• Comfortable living situation

• Adequate and affordable health services

• Adequate and affordable food

• Affordable community recreational activities

• Family and friend financial assistance

56


Sample Items:

• I can afford to eat out when I want.

• I can afford to pay my bills.

• I have delayed getting health care because I couldn’t pay for it.

• I have access to extra money in case of an emergency.

57

Systems, Services, and Policies

Affect on participation:• Home• Community• Work

Five sub-sets:•Managing health•Living situation•Community participation•Work •Transportation

58

Systems, Services, and Policies

Sample Items:

• My community offers support groups I can use.

• My community offers information on low or no cost activities and things to do.

• Disability accommodation services and supports are available at my work or school.

• Public transportation in my area is reliable.

59


Features of: • Home• Buildings• Outdoors

How much difficulty do you have...?• Environmental ?• Non-environmental ?

60


Sample Items:

• How much difficulty would you have feeling safe in your home during an emergency?

• The difficulty I have feeling safe is due to problems getting out or getting help in an emergency. (yes – no)

• How much difficulty do you have hearing sounds such as voices and music in buildings in your community?

• The difficulty I have hearing sounds is due to background noise. (yes – no)

61


Four sub-sets:• Mobility• Personal care• Vision/Hearing

• Communication

Includes:• Availability• Quality• Impact

62

http://www.google.com/imgres?imgurl=http://www.access.duke.edu/images/landing/land_assistive-tech.jpg&imgrefurl=http://www.access.duke.edu/assistive_technology/index.php&usg=__HQIuIaUslucobvawGD6cX1EHlzs=&h=190&w=230&sz=5&hl=en&start=245&zoom=0&itbs=1&tbnid=p00CMEBhO9nfxM:&tbnh=89&tbnw=108&prev=/search%3Fq%3Dassistive%2Btechnology%26start%3D231%26hl%3Den%26sa%3DN%26biw%3D1580%26bih%3D719%26ndsp%3D21%26tbm%3Disch%26prmd%3Divnslb&ei=EOPwTe_fLcff0QGmsdHKAw

http://www.google.com/imgres?imgurl=http://www.wramc.amedd.army.mil/NewsAndEvents/media/wrimages/11-12-2010/450assistive%2520tech%2520magnifier.jpg&imgrefurl=http://www.wramc.amedd.army.mil/NewsAndEvents/media/resource/Lists/wrarticles/DispForm.aspx%3FID%3D708&usg=__7k6ZcJnjcbf5i0_gXdXEugaqASM=&h=293&w=450&sz=772&hl=en&start=59&zoom=1&itbs=1&tbnid=KmcB1gK14q_KqM:&tbnh=83&tbnw=127&prev=/search%3Fq%3Dassistive%2Btechnology%2Bvision%26start%3D42%26hl%3Den%26sa%3DN%26biw%3D1580%26bih%3D719%26ndsp%3D21%26tbm%3Disch&ei=4-PwTdOzL4rl0QGnq4jcAw


Sample Items:

• My device is easy to use.

• My device is reliable.

• My device gives me more control over my daily activities.

• My device allows me to participate in activities that I enjoy.

63

Access to Information & Technology

• Devices and technology to transmit and receive information– cell phone, land lines, computer, email, and

internet services

• Usability of this information/technology– ability to access and understand information,

literacy, transparency, information finding

64

Sample Items

• I have easy access to the internet if I want to use it.

• My telephone is easy to use.

• Information about community resources is easily available to me if I need it.

• In case of a health emergency I can get the information I need easily.

65

Access to Information & Technology

Social Environment

Social Support:• Companionship• Emotional Support• Informational Support• Instrumental Support

Perceived Positive & Negative Disability-related Attitudes & Behaviors: From:

• Family/Friends• Community• Society / Public

Including:• Acceptance• Stigma• Marginalization

66

Social Environment

Sample Items:

• The people in my life are willing to accommodate my disability.

• People are patient when I take extra time to do things because of my disability.

• Criminals see people with disabilities as easy targets.

• Because of my disability, my family complains that I am too needy.

• Society respects the need for disability accommodations.

• Society limits my opportunities because of my disability.

• People with disabilities are encouraged to participate in my community.

67

Cognitive Interviewing

• 5 people per impairment group• Interviews were recorded for transcription

purposes• Follow-up questions were asked

intentionally after each question• Comprehension• Opinions• Recommendations

68

My doctors, nurses and other health care professionals give me written information in ways I can understand.

Health care providers’ elicited too narrow a response; ‘Doctors and nurses’ named explicitly.

My health care providers give me written information in ways I can understand.

69





70





71



In case of a natural disaster, I can get the information I need easily.

In case of a health emergency, I can get the information I need easily.

‘Emergency’ was too broad; broke item up into personal health/environmental.

In case of an emergency, I can get the information I need easily.

72






73






74

Preliminary Pilot Test Results

75

Domain/Rating Scale Categories

N of Items

Reliability(Criterion <.8)

Misfitting Items Weakly Correlated Items

Residual Variance

(Criterion <10%)

CFA Results

Access to Information & Technology/4

25 .82 10 fax11 phone easy

11 phone easy use12 phone reliable17 radio available18 radio reception

10.2% Multidimensional

Assistive Technology-Mobility/5

14 .73 9 Replace device None 12.2% Multidimensional

Assistive Technology-Personal Care/4

14 .66 26 Variety places None 15.0% Multidimensional

Built & Natural/5

18 .72 None 4 conversations @ home

15 hear sounds outdoors

7.6% Good

Preliminary Pilot Test Analyses, cont.

76

Domain/Rating Scale Categories

N of Items

Reliability(Criterion

<.8)

Misfitting Items

Weakly Correlated Items

Residual Variance

(Criterion <10%)

CFA Results

Systems Services Policies: Health/4

11 .72 None 3 Emergency medical

11.2% Pending

Systems Services Policies: Home/4

20 .84 22 Food None 9.4% Pending

Systems Services Policies: Community/4

11 .85 37 Legal None 7.2% Pending

Systems Services Policies: Work, Learn, 4

12 .75.

43 Library51 SSDI

43 Library 9.4% Pending

Systems Services Policies: Transport/4

11 .74 57 Air travel58 Car access

None 12.9% Pending

Systems Services Policies: Alt Transport/4

8 .59 None None 11.7% Pending

Preliminary Pilot Test Analyses, cont.

77

Domain/ Rating Scale Categories

N of Items

Reliability(Criterion <.8)

Misfitting ItemsWeakly

Correlated Items

Residual Variance

(Criterion <10%)

CFA Results

Social Environment: Personal/4

26 .95 24 Encouraged32 Criminals34 “Special”63 Unhelpful64 Public

32 Criminals64 Public

5.7% Pending

Social Environment: Societal/4

56 .94 70 Criminals78 Underestimates

70 Criminals 8.2% Pending

Economic Quality of Life/5

37 .92 17 Meds26 Utilities33 6 months34 Improve36 $ important

13 HVAC17 Meds26 Utilities34 Improve36 $ important

4.8% 0.1

ECON QOL DATA?

78

Relevance to TBI

79

80

Next Steps

• Complete Data Analysis– Explore dimensionality of domains– Explore differential item functioning by impairment

group

• Field Testing– Recruit 200 participants with TBI, 200 with Stroke, and

200 with SCI (N=600)– Administer EF items banks, legacy measures, newly

developed social participation measures (TBI-QOL) and traditional neuropsychological measures

Acknowledgments

Rehabilitation Research and Training Center on Improving Measurement of Medical Rehabilitation Outcomes (H133B090024)

Rehabilitation Research and Training Center on Measuring Rehabilitation Outcomes and Effectiveness (H133B040032)

81

Participation and Environmental Factors Measures For TBI Rehabilitation

Documents

Transcript of Participation and Environmental Factors Measures For TBI Rehabilitation