2015 - A Rasch Analysis of the SIS
1
The purpose of this study is to investigate the psychomet- ric properties of the Spiritual Identity Scale-Revised (SIS-R; Zamora & Richards, 2014) using Partial Credit Model Item Response Theory on a sample of 310 undergraduate stu- dents at Brigham Young University (187 females [60.3%] and 123 males [39.7%]; mean age = 22.3 [mode: 21]). The SIS-R is a 41-item instrument designed to assess an individual’s endorsement and aspiration of various spir- itual and secular values. As a means to evaluate self-es- teem, the SIS-R measures self-acceptance, self-criti- cism, and self-worth. In total, the SIS-R is composed of 2 subtests with a total of 6 subscales. The SIS-R is based on a theistic view of healthy identity and self-esteem, which posits that spiritually mature peo- ple will tend to believe in their inherent worth, indepen- dent of their achievements and external appearance. This view hypothesizes that spiritually mature people will base their feelings of self-esteem on spiritual traits and char- acteristics (e.g., being honest, prayerful, forgiving, moral, etc.) rather than on materialistic or “worldly” traits and characteristics (e.g., being wealthy, sexy, beautiful, popu- lar, etc.; see Richards, 2000; Richards & Bergin, 1997). a rasch analysis of the Spiritual identity scales P. Scott Richards, PhD Counseling Psychology Professor, Faculty Mentor Justin P. Zamora, MS Counseling Psychology Doctoral Student RESULTS RESULTS (CONT’D) Winsteps (t. 3.80; Linacre, 2013) was used to analyze the scales of the SIS-R. We employed the Rating Scale Model (RSM; Andersen, 1997), a polytomous extension of the di- chotomous Rasch model in order to account for the Likert- scale responses of the SIS-R. IRT models provide various reliability, fit, and summary statis - tics that may be unfamiliar as this approach is relatively new in the literature (Funk & Rogge, 2007). In order to help with interpretation, a brief review of IRT statistics is provided. •Reliability statistics cover both person reliability and item reliability. Person reliability evaluates the ability of the scale to differentiate participants on the measured la - tent trait and the replicaiblity of the placement of partic- ipants on that latent continuum (Wright & Masters, 1982). Item reliability indicates the width of a measure’s cover- age; the lower the value of the item reliability, the narrow- er the range of the latent trait continuum being measured. •Separation (person or item) indicates an estimate of the spread of persons (or items) on the measured latent contin- uum (Wright & Masters, 1982). Person (or item) separation has a range from zero to infinity, and ideally should be at least 1.25,whichindicatesthatthescaleseparatespersonsoritems (or both) into at least two distinct groups (Linacre, 2013). •Summary statistics , called Rasch fit statistics, were used to asses how well the data fit the model. The outfit and in - fit statistics are typically reported as both mean-square fit statistics (MNSQ) and as z-standardized fit statistics (ZSTD). MNSQ values between 0.5 and 1.5 indicate good fit (Linacre, 2013). MNSQ indicates the level of random - ness in the data. For ZSTD, +2.0 (i.e., 2 SD away from the mean, 0.0) is often used as cutoff values for good fit. Jabra F. GHNEIM, MS EDucational inquiry, measurement, & Evaluation doctoral student Richard R. Sudweeks, PHD EDucational inquiry, measurement, & Evaluation Professor, faculty mentor overview methodology INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I am religious -1.35 0.83 -1.8 0.69 -2.5 I am prayerful 0.83 1.01 0.1 1.01 0.2 I am moral -0.26 1.26 2.7 1.24 2.2 I am spiritual 0.78 0.87 -1.5 0.83 -1.9 DEVOUT BEHAVIOR Person Item Separation 1.64 5.97 Reliability 0.73 0.98 Infit MNSQ 0.98 0.99 Infit ZSTD -.01 -0.11 Outfit MNSQ 0.94 0.94 Outfit ZSTD -0.1 -0.5 0 1 2 3 -1 -2 -3 religious prayerful moral spiritual 0 1 2 3 -1 -2 -3 intelligent influential powerful successful INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I am powerful 1.17 0.97 -0.3 0.99 0.0 I am influential 0.89 1.08 1.0 1.04 0.5 I am successful -0.36 0.84 -2.0 0.82 -2.2 I am intelligent -1.70 1.05 0.7 1.01 0.1 social status Person Item Separation 1.79 9.88 Reliability .76 .99 Infit MNSQ .98 .99 Infit ZSTD -.2 -.2 Outfit MNSQ .96 .96 Outfit ZSTD -.1 -.4 0 1 2 3 -1 -2 -3 sexy athletic physically fit shapely INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I am physically fit -0.40 0.73 -3.7 0.70 -4.1 I am athletic -0.32 1.19 1.7 1.14 1.7 I am shapely -0.06 .95 -0.5 0.95 -0.5 I am sexy 0.79 1.13 1.4 1.12 1.4 body image Person Item Separation 2.00 4.57 Reliability .80 .95 Infit MNSQ .98 1.00 Infit ZSTD -.2 -.1 Outfit MNSQ .98 .98 Outfit ZSTD -.2 -.4 INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I am patient 1.08 0.90 -1.3 0.90 -1.2 I am tolerant 0.51 0.84 -2.0 0.84 -1.9 I am forgiving -.19 0.91 -1.0 -.91 -1.0 I am loving -1.40 1.33 3.5 1.36 3.4 Relating to others Person Item Separation 1.46 8.18 Reliability .68 .99 Infit MNSQ .98 .99 Infit ZSTD -.1 -.2 Outfit MNSQ 1.01 1.01 Outfit ZSTD -.1 -.2 0 1 2 3 -1 -2 -3 theta (θ) patient tolerant forgiving loving INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I would like to be moral -.60 .72 -2.1 .65 -2.2 I would like to be religious -.41 .86 -.9 .95 -.2 I woud like to be spiritual -.05 .73 -1.7 .72 -1.3 I would like to be prayerful .48 1.09 .6 1.27 1.4 I would like to be tolerant .41 1.30 2.3 1.27 2.2 I would like to be patient -.28 1.00 .1 .98 -.1 I would like to be forgiving .44 1.13 2.3 1.27 2.2 DESIRED SpirituaL Self Person Item Separation .25 2.20 Reliability .06 .83 Infit MNSQ 1.02 .98 Infit ZSTD .1 -.1 Outfit MNSQ .99 .99 Outfit ZSTD .0 .1 INFIT OUTFIT Item θ MNSQ ZSTD MNSQ ZSTD I would like to be intelligent -1.43 .81 -1.8 .73 -1.5 I would like to be influential .70 .94 -.7 .90 -1.2 I woud like to be successful -1.36 .98 -.1 .80 -1.1 I would like to be powerful 1.83 1.27 3.0 1.28 3.2 I would like to be sexy 1.35 1.08 1.0 1.09 1.1 I would like to be shapely -.10 .84 -1.9 .85 -1.5 I would like to be physically fit -1.34 .98 -.1 .80 -1.1 I would like to be athletic .35 1.180 1.9 1.21 2.2 DESIRED secular Self Person Item Separation 1.43 10.20 Reliability .67 .99 Infit MNSQ 1.04 .98 Infit ZSTD .1 -.1 Outfit MNSQ .94 .94 Outfit ZSTD .0 -.1 0 1 2 3 -1 -2 -3 moral forgiving prayerful religious tolerant spiritual patient 0 1 2 3 -1 -2 -3 successful influential intelligent powerful sexy shapely physically fit athletic RASCH FIT STATISTICS Presented below are the results of the IRT analysis with each subscales respective Rasch fit statistics. In general, the SIS-R subscales indicate adequate person and item reliability and separation estimates for Devout Behav- ior, Relating to Others, Social Status, Body Image, and Desired Secular Self. Person reliability and separation estimates for Desired Spiritual Self were considerably low indicating difficulty in distinguishing individuals de- spite adequate item reliability and separation estimates. TEST INFORMATION CURVES The IRT analyses also provide important test information curves that give an estimation of both the amount and range of information each scale provides on the latent constructs being measured. In general, the more information underneath an information curve, the better (Funk & Rogge, 2007). Be- low the test information curves for the 7 SIS-R scales . These results indicate a moderate area of coverage for the first 5 scales and leptokurtotic coverage for both the Desired Spir- itual Self and Desired Secular Self subscales. VARIABLE MAP To the right of each test information curve is a represen- tation of each item on the latent variable continuum (Bond & Fox, 2001). The horizontal axis represents item difficulty ( θ), with lower values indicating “easier” items, higher values in - dicated “harder” items, and 0 indicating average items. The ver- tical axis assesses the fit of each item on the latent construct with ± 1.5 indicating acceptable fit. ± Overerfit Underrfit 2 -2 Overerfit Underrfit 2 -2 Overerfit Underrfit 2 -2 Overerfit Underrfit 2 -2 Overerfit Underrfit 2 -2 Overerfit Underrfit 2 -2 SCALE CORRELATIONS Moderate correlations were found between Devout Behavior, Relating to Others, and Social Status; between Ideal Secular Self and Social Status, and Body Image. Strong correlations were found between Social Status and Body Image, which is to be expected. Surprisingly, Ideal Secular Self and Ideal Spiritual Self, which may indicate that respondents aspire to improve or enhance their secular identity just as much as their spiritual. DISCUSSION references Andersen, E. B. (1997). The rating scale model. In Handbook of modern item response theory (pp. 67-84). New York: Springer. Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model: Fundamental mea- surement in the human sciences. New York: Psychology Press. Linacre, J. M. (2013). Winsteps 3.80. 1 [computer software]. Chicago, IL: Win- steps. com. Funk, J. L., & Rogge, R. D. (2007). Testing the ruler with item response theo- ry: increasing precision of measurement for relationship satisfaction with the Couples Satisfaction Index. Journal of Family Psychology, 21(4), 572. Bergin, A. E., & Richards, P. S. (1997). A spiritual strategy for counseling and psychotherapy. Washington, DC: American Psychological Association. Richards, P. S., & Bergin, A. E. (2000). Toward religious and spiritual compe- tency for mental health professionals. Washington, DC: American Psycho- logical Association. 1 2 3 4 5 1 Devout Behavior — 2 Relating to Others .303** — 3 Social Status .397** .313** — 4 Body Image .249** .222** .521** — 5 Ideal Spiritual Self .296** .067 .105 .033 — 6 Ideal Secular Self .059 .016 .368** .306** .423** ** p<.01 The purpose of this study was to conduct a Rasch analysis of the SIS-R using a sample of religious undergraduate students who attend Brigham Young University. Results indicate that items within each subscales fit the latent variable well and possess strong reliability and fit estimates. Several key findings are instructive to spiritual and religious re- searchers and practitioners. First, items related to adherence to spiritual practice (e.g. I am prayerful) were more difficult than items related to religiosity. Second, items related to engaging with others who may be difficult (e.g. I am patient; I am toler- ant) were more difficult than items relating to engaging with pleasant people (e.g. I am loving). Third, items relating exerting power over others (e.g. I am influential) were considerably more difficult than social comparisons (e.g. I am intelligent). Together, these findings have several clinical implications. First, assessing a client’s spiritual practice may provide more relevant information than inquiring about religiousness (e.g. attendance, belief system). Second, understanding how clients interact and cope with others in their lives may provide pertinent information regarding their well-being. Lastly, in seeking to improve them- selves, clients may simultaneously pursue to enhance their spir- itual identity simultaneously with their secular identity. Iden- tifying healthy methods to achieve both may be something to encourage rather than dissuade.
-
Upload
justin-zamora -
Category
Documents
-
view
10 -
download
4
Transcript of 2015 - A Rasch Analysis of the SIS
The purpose of this study is to investigate the psychomet-ric properties of the Spiritual Identity Scale-Revised (SIS-R; Zamora & Richards, 2014) using Partial Credit Model Item Response Theory on a sample of 310 undergraduate stu-dents at Brigham Young University (187 females [60.3%] and 123 males [39.7%]; mean age = 22.3 [mode: 21]). The SIS-R is a 41-item instrument designed to assess an individual’s endorsement and aspiration of various spir-itual and secular values. As a means to evaluate self-es-teem, the SIS-R measures self-acceptance, self-criti-cism, and self-worth. In total, the SIS-R is composed of 2 subtests with a total of 6 subscales.The SIS-R is based on a theistic view of healthy identity and self-esteem, which posits that spiritually mature peo-ple will tend to believe in their inherent worth, indepen-dent of their achievements and external appearance. This view hypothesizes that spiritually mature people will base their feelings of self-esteem on spiritual traits and char-acteristics (e.g., being honest, prayerful, forgiving, moral, etc.) rather than on materialistic or “worldly” traits and characteristics (e.g., being wealthy, sexy, beautiful, popu-lar, etc.; see Richards, 2000; Richards & Bergin, 1997).
a rasch analysis of the Spiritual identity scalesP. Scott Richards, PhD
Counseling PsychologyProfessor, Faculty Mentor
Justin P. Zamora, MSCounseling Psychology
Doctoral Student
RESULTS RESULTS (CONT’D)
Winsteps (t. 3.80; Linacre, 2013) was used to analyze the scales of the SIS-R. We employed the Rating Scale Model (RSM; Andersen, 1997), a polytomous extension of the di-chotomous Rasch model in order to account for the Likert-scale responses of the SIS-R.IRT models provide various reliability, fit, and summary statis-tics that may be unfamiliar as this approach is relatively new in the literature (Funk & Rogge, 2007). In order to help with interpretation, a brief review of IRT statistics is provided. • Reliability statistics cover both person reliability and
item reliability. Person reliability evaluates the ability of the scale to differentiate participants on the measured la-tent trait and the replicaiblity of the placement of partic-ipants on that latent continuum (Wright & Masters, 1982). Item reliability indicates the width of a measure’s cover-age; the lower the value of the item reliability, the narrow-er the range of the latent trait continuum being measured.
• Separation (person or item) indicates an estimate of the spread of persons (or items) on the measured latent contin-uum (Wright & Masters, 1982). Person (or item) separation has a range from zero to infinity, and ideally should be at least 1.25, which indicates that the scale separates persons or items (or both) into at least two distinct groups (Linacre, 2013).
• Summary statistics, called Rasch fit statistics, were used to asses how well the data fit the model. The outfit and in-fit statistics are typically reported as both mean-square fit statistics (MNSQ) and as z-standardized fit statistics (ZSTD). MNSQ values between 0.5 and 1.5 indicate good fit (Linacre, 2013). MNSQ indicates the level of random-ness in the data. For ZSTD, +2.0 (i.e., 2 SD away from the mean, 0.0) is often used as cutoff values for good fit.
Jabra F. GHNEIM, MSEDucational inquiry,
measurement, & Evaluationdoctoral student
Richard R. Sudweeks, PHDEDucational inquiry,
measurement, & EvaluationProfessor, faculty mentor
overview
methodology
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I am religious -1.35 0.83 -1.8 0.69 -2.5I am prayerful 0.83 1.01 0.1 1.01 0.2I am moral -0.26 1.26 2.7 1.24 2.2I am spiritual 0.78 0.87 -1.5 0.83 -1.9
DEVOUTBEHAVIOR
Person ItemSeparation 1.64 5.97Reliability 0.73 0.98Infit MNSQ 0.98 0.99Infit ZSTD -.01 -0.11Outfit MNSQ 0.94 0.94Outfit ZSTD -0.1 -0.5
0 1 2 3-1-2-3
religious
prayerful
moral
spiritual
0 1 2 3-1-2-3
intelligent influential
powerful
successful
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I am powerful 1.17 0.97 -0.3 0.99 0.0I am influential 0.89 1.08 1.0 1.04 0.5I am successful -0.36 0.84 -2.0 0.82 -2.2I am intelligent -1.70 1.05 0.7 1.01 0.1
social status
Person ItemSeparation 1.79 9.88Reliability .76 .99Infit MNSQ .98 .99Infit ZSTD -.2 -.2Outfit MNSQ .96 .96Outfit ZSTD -.1 -.4
0 1 2 3-1-2-3
sexyathletic
physically fit
shapely
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I am physically fit -0.40 0.73 -3.7 0.70 -4.1I am athletic -0.32 1.19 1.7 1.14 1.7I am shapely -0.06 .95 -0.5 0.95 -0.5I am sexy 0.79 1.13 1.4 1.12 1.4
body image
Person ItemSeparation 2.00 4.57Reliability .80 .95Infit MNSQ .98 1.00Infit ZSTD -.2 -.1Outfit MNSQ .98 .98Outfit ZSTD -.2 -.4
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I am patient 1.08 0.90 -1.3 0.90 -1.2I am tolerant 0.51 0.84 -2.0 0.84 -1.9I am forgiving -.19 0.91 -1.0 -.91 -1.0I am loving -1.40 1.33 3.5 1.36 3.4
Relating to others
Person ItemSeparation 1.46 8.18Reliability .68 .99Infit MNSQ .98 .99Infit ZSTD -.1 -.2Outfit MNSQ 1.01 1.01Outfit ZSTD -.1 -.2
0 1 2 3-1-2-3theta (θ) patient
tolerantforgiving
loving
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I would like to be moral -.60 .72 -2.1 .65 -2.2I would like to be religious -.41 .86 -.9 .95 -.2I woud like to be spiritual -.05 .73 -1.7 .72 -1.3I would like to be prayerful .48 1.09 .6 1.27 1.4I would like to be tolerant .41 1.30 2.3 1.27 2.2I would like to be patient -.28 1.00 .1 .98 -.1I would like to be forgiving .44 1.13 2.3 1.27 2.2
DESIREDSpirituaL Self
Person ItemSeparation .25 2.20Reliability .06 .83Infit MNSQ 1.02 .98Infit ZSTD .1 -.1Outfit MNSQ .99 .99Outfit ZSTD .0 .1
INFIT OUTFITItem θ MNSQ ZSTD MNSQ ZSTD
I would like to be intelligent -1.43 .81 -1.8 .73 -1.5I would like to be influential .70 .94 -.7 .90 -1.2I woud like to be successful -1.36 .98 -.1 .80 -1.1I would like to be powerful 1.83 1.27 3.0 1.28 3.2I would like to be sexy 1.35 1.08 1.0 1.09 1.1I would like to be shapely -.10 .84 -1.9 .85 -1.5I would like to be physically fit -1.34 .98 -.1 .80 -1.1I would like to be athletic .35 1.180 1.9 1.21 2.2
DESIREDsecular Self
Person ItemSeparation 1.43 10.20Reliability .67 .99Infit MNSQ 1.04 .98Infit ZSTD .1 -.1Outfit MNSQ .94 .94Outfit ZSTD .0 -.1
0 1 2 3-1-2-3
moral
forgiving
prayerful
religious
tolerant
spiritualpatient
0 1 2 3-1-2-3 successfulinfluentialintelligent
powerful
sexy
shapely
physically fit
athletic
RASCH FIT STATISTICSPresented below are the results of the IRT analysis with each subscales respective Rasch fit statistics. In general, the SIS-R subscales indicate adequate person and item reliability and separation estimates for Devout Behav-ior, Relating to Others, Social Status, Body Image, and Desired Secular Self. Person reliability and separation estimates for Desired Spiritual Self were considerably low indicating difficulty in distinguishing individuals de-spite adequate item reliability and separation estimates.
TEST INFORMATION CURVESThe IRT analyses also provide important test information curves that give an estimation of both the amount and range of information each scale provides on the latent constructs being measured. In general, the more information underneath
an information curve, the better (Funk & Rogge, 2007). Be-low the test information curves for the 7 SIS-R scales . These results indicate a moderate area of coverage for the first 5 scales and leptokurtotic coverage for both the Desired Spir-itual Self and Desired Secular Self subscales.
VARIABLE MAPTo the right of each test information curve is a represen-tation of each item on the latent variable continuum (Bond & Fox, 2001). The horizontal axis represents item difficulty (θ), with lower values indicating “easier” items, higher values in-dicated “harder” items, and 0 indicating average items. The ver-tical axis assesses the fit of each item on the latent construct with ± 1.5 indicating acceptable fit.
±
Ove
rerfi
tUn
derr
fit
2
-2
Ove
rerfi
tUn
derr
fit
2
-2
Ove
rerfi
tUn
derr
fit
2
-2
Ove
rerfi
tUn
derr
fit
2
-2
Ove
rerfi
tUn
derr
fit
2
-2
Ove
rerfi
tUn
derr
fit
2
-2
SCALE CORRELATIONSModerate correlations were found between Devout Behavior, Relating to Others, and Social Status; between Ideal Secular Self and Social Status, and Body Image. Strong correlations were found between Social Status and Body Image, which is to be expected. Surprisingly, Ideal Secular Self and Ideal Spiritual Self, which may indicate that respondents aspire to improve or enhance their secular identity just as much as their spiritual.
DISCUSSION
referencesAndersen, E. B. (1997). The rating scale model. In Handbook of modern item
response theory (pp. 67-84). New York: Springer.Bond, T. G., & Fox, C. M. (2013). Applying the Rasch model: Fundamental mea-
surement in the human sciences. New York: Psychology Press.Linacre, J. M. (2013). Winsteps 3.80. 1 [computer software]. Chicago, IL: Win-
steps. com.Funk, J. L., & Rogge, R. D. (2007). Testing the ruler with item response theo-
ry: increasing precision of measurement for relationship satisfaction with the Couples Satisfaction Index. Journal of Family Psychology, 21(4), 572.
Bergin, A. E., & Richards, P. S. (1997). A spiritual strategy for counseling and psychotherapy. Washington, DC: American Psychological Association.
Richards, P. S., & Bergin, A. E. (2000). Toward religious and spiritual compe-tency for mental health professionals. Washington, DC: American Psycho-logical Association.
1 2 3 4 51 Devout Behavior —2 Relating to Others .303** —3 Social Status .397** .313** —4 Body Image .249** .222** .521** —5 Ideal Spiritual Self .296** .067 .105 .033 —6 Ideal Secular Self .059 .016 .368** .306** .423**** p<.01
The purpose of this study was to conduct a Rasch analysis of the SIS-R using a sample of religious undergraduate students who attend Brigham Young University. Results indicate that items within each subscales fit the latent variable well and possess strong reliability and fit estimates. Several key findings are instructive to spiritual and religious re-searchers and practitioners. First, items related to adherence to spiritual practice (e.g. I am prayerful) were more difficult than items related to religiosity. Second, items related to engaging with others who may be difficult (e.g. I am patient; I am toler-ant) were more difficult than items relating to engaging with pleasant people (e.g. I am loving). Third, items relating exerting power over others (e.g. I am influential) were considerably more difficult than social comparisons (e.g. I am intelligent).Together, these findings have several clinical implications. First, assessing a client’s spiritual practice may provide more relevant information than inquiring about religiousness (e.g. attendance, belief system). Second, understanding how clients interact and cope with others in their lives may provide pertinent information regarding their well-being. Lastly, in seeking to improve them-selves, clients may simultaneously pursue to enhance their spir-itual identity simultaneously with their secular identity. Iden-tifying healthy methods to achieve both may be something to encourage rather than dissuade.
