1

The Pilot Phase in the Development of the

Multicultural Awareness Observer Rating ScaleGargi Roysircar, Connie L. Bates, Susan G. Danielson, Nicole C. Spanakis, Natalia S. Duke, Dawn M. Brock,

Catherine M. Iacuzzi, Sarah Smith, and Sarah McLaughlin

Antioch New England Graduate School

2

Description of a Community Activity in Multicultural Training

The Multicultural Interactions Project is an experiential component of a course, called Human Diversity and the Clinical Enterprise, taught in a doctoral clinical psychology program in the Northeast. Students are asked to either interview an individual or engage in a service learning project with an individual who is culturally different from them. This activity is conducted within one’s local community. Students’ interviewees may be diverse based on age, class, culture, race, ethnicity, national origin, sexual identity, or disability status. These meetings are explicitly not therapeutic in nature, but rather a mutual learning opportunity through conversations and friendly support. The purpose of these meetings is for clinical trainees to gain awareness of (a) their own assumptions, values, and biases, and (b) the worldviews of culturally different individuals.

3

Components of the Multicultural Interactions Project

• 10 meetings (approximately 60 minutes each) with an individual who is culturally diverse from the student

• Weekly process notes submitted to the professor based on these meetings

• Pre- and post-test evaluation questionnaires for the student and interviewee

• A final case study paper, summarizing the student’s learning

SampleIt is expected that data from 100 students who voluntarily and anonymously contribute the above-mentioned course materials for research will be analyzed for an outcome study in training.

4

Process Notes• Students are asked to submit weekly process notes to the instructor

• These process notes are meant to be an opportunity for the student to examine his or her experience of meeting with an interviewee

• Students are asked to focus on process issues (dyadic interactions and trainee intrapersonal issues) within a meeting session

• Students write self-reflections in the process notes

• See Roysircar, Gard, Hubbell, & Ortega (2004) for more information

5

Method: Training of CodersThe coders held one-hour weekly meetings from January through July of 2004 for a total of 22 hours of training. They received training in the connection and disconnection themes developed from analysis of process notes from 67 counselor trainees at the University of Nebraska-Lincoln (Roysircar, Gard, Hubbell, & Ortega, 2004). The training involved understanding and identifying the themes in process notes from clinical psychology students at Antioch New England Graduate School. The meetings also provided an opportunity for peer consultation on the themes and the coding for sample process notes. Some consultations were also held on an individual basis outside of the meetings.

6

Method: Rating of ThemesThe coders kept a frequency count of each theme for each process note to determine the presence of the themes and to gain proficiency in recognizing the themes. Coders also tracked the reported client responses to the use of a theme in a session. A rating system was developed to convert frequency counts to a 5-point Likert scale. The scoring reflected the presence or absence of the themes, as well as any response by the client. The scale used was as follows: 1 = theme not present; 2 = one theme with no client response; 3 = the same theme twice without a client response or 1 theme with a client response; 4 = 3 or more times the same theme without a client response or the same theme twice with a client response; and 5 = 3 or more times of the same theme with a client response. A separate Likert scale was developed to rate the strength or degree of emphasis placed on interventions that the trainee reported in a process note. The scale developed is as follows: 1 = weak; 2 = mild; 3 = moderate; 4 = strong; 5 = very strong.

7

Interrater Reliability Each coder applied the rating system to eight sample process notes, which were then analyzed for interrater reliability.

Preliminary Exploratory AnalysisEach coder also rated additional process notes, with a total N=200 (20 cases x 10 process notes), which were analyzed for preliminary statistical evidence.

8

Interrater Reliability

Table 2. Interrater Reliability Rater Agreement Kappa

 Rater 1 and 5 .89 .67Rater 4 and 5 .85 .61 Rater 2 and 3 .81 .55Note. N = 80 process notes. For each process note, each rater identified 10 themes on a Likert scale of 1 = Theme not present; 2 = 1 theme present with no client response; 3 = 2 themes present without a clientresponse or 1 theme present with a client response; 4 = 3 or more themes present without a client response or 2 themes present with a client response; 5 = 3 or more themes present with a client response.

Table 1. Pearson Correlations of Pilot Ratings by Five RatersRaters 1 2 3 4 5 Rater 1 (1) -- .69** .69** .67** .75**Rater 2 (2) -- .68** .58** .60**Rater 3 (3) -- .59** .67**Rater 4 (4) -- .69**Rater 5 (5) --Note. N = 80 process notes. ** p<.01 (2-tailed)

9

Table 3. Cronbach’s Alpha of Ratings for Each Theme

Cronbach’s Alpha

Connection VariablesC 1: Differences Integrated .71C 2: Cultural Empathy Cognitive .76C 3: Affective Connection with Client’s Culture .77C 4: Counselor Self-Disclosure .84C 5: Counselor Self-Reflection .52Disconnection VariablesD 1: Environmental Barriers 1.00D 2: Frustration with Environmental Barriers 1.00D 3: Unintegrated Differences .76D 4: Overgeneralization of Similarities .38D 5: Stereotypes .93

Note. N = 80 process notes. Total Cronbach’s alpha for Connection = .94. Total Cronbach’s alpha for Disconnection = .81. Correlation between Connection and Disconnection = -.75. In a separate analysis of Intervention items, the Cronbach’s alpha for the Intervention dimension was .61.

10

Table 4. Factor Loadings of Connection, Disconnection, and Intervention Items 

Items Factors Cultural Info Cultural Dyadic Empathy

Self-Awareness Interface_________________________________________________________________________________________________Int. 3: Information exchange on customs and values .63Int. 4: Talking about family (members, behaviors, practices) .65Int. 8: Power differential and equalizing power .24Int. 9: Client acculturation .43Int. 10: Client’s racism experience .61Int. 11: Client and counselor worldviews .46Int. 17: Client tells a story .53D 1: Environmental barriers in counseling .23*D 4: Overgeneralization of similarities .38C 4: Counselor self-disclosure .65C 5: Counselor self-reflection .67D 5: Stereotypes .53Int. 12: Discussion of comfort level .54C 1: Differences integrated .71D 2: Frustrations with environmental barriers .49*Int. 13: Taking a chance/risk .47*C 2: Cultural empathy cognitive .59C 3: Affective connection with client’s culture .58D 3: Unintegrated differences .53*

Eigenvalues 14.40 9.76 7.24 6.72% Variance Explained 12.37 10.82 7.74 7.20

Note. N = 197 process notes. * Reverse scored. Extraction method: Principal Component Analysis with varimax rotation. Total variance explained was 38.12%.

11

Figure 1. Examples From Process Notes Illustrating the

Themes Connection and Disconnection and Intervention Items

Note. The Connection/Closeness and Disconnection/Distance themes were previously derived qualitatively in a study on self-reflection process notes on mentoring ESL students (Roysircar, Gard, Hubbell, & Ortega, 2004). The intervention items were piloted in the current preliminary study after an initial reading of several hundred process notes.

Connection/Closeness 

C1 Differences Integrated

She helped me to see my culture from a new perspective and to begin to think about which cultural values and practices I want to adopt and those I want to reject. That is the beauty of our interactions. While we are learning more about each other we are also discovering new things about our cultures and ourselves.

 

One of the differences we explored was the safety of the environments where we grew up. A was born and raised in Sao Paulo. She described that she lived with a certain vigilance concerning her physical safety and of those close to her. Although she lived in a safe neighborhood, it was surrounded with more turbulent and dangerous ones. A described that she and her family always considered their personal safety because theft, kidnapping, and violence were (and are) commonplace activities. A. explained that the degree of violence is related to the resentment between the poor and wealthy classes and may be an expression of an oppressed and disenfranchised people. I shared with A. what it was like for me growing up in New York City. Although the city was never considered 'secure' (this was pre-Giulianni) and I was mugged numerous times, I grew up with a general feeling of safety while out and about in the streets. I was aware of my environment and of a general sense of my personal safety but I did not need to be too vigilant about it. There was (and is) a marked divide between the wealthy and the poor, but it did not manifest in acts of unpredictable violence as it appeared to in Brazil.

12

C2 Cultural Empathy Cognitive I asked C. what her experience was like during the Hurricane Floyd, knowing that she was from Greenville, an area that was severely hit by flooding. C. seemed almost surprised that someone from this area actually knew that Greenville was flooded.

The prayer on the card was in Spanish, and I asked her to translate it for me, which she did, teaching me some words in the process.

C3 Affective ConnectionI mentioned how I think she did a wonderful deed by caring for her siblings to help her mother out. She smiled and nodded in appreciation.

I expressed my feelings to L, of how difficult the situation must have been for her.

13

C4 Counselor Self-Disclosure

I definitely felt a little awkward in the first few minutes; having virtually no information about H, other than she had difficulty speaking English.

I shared my personal experience with S about living in the Middle East.

 

C5 Counselor Self-Reflection

I explained to P that my reaction was due to the different ways that our cultures viewed abuse. She said she understood but I still felt like I had missed out on a good chance to be empathetic and improve the helping relationship. I let my own cultural beliefs get in the way of understanding her experiences.

14

Disconnection/Distance 

D1 Environmental Barriers

We did talk more about practicing telephone skills. I had called her a couple of times. She had not called me so I decided that perhaps she was not truly interested in the phone calls.  D2 Frustration with Environmental Barriers

None. D3 Unintegrated differences

When P told me this I laughed in disbelief. Coming from my own experience of being taught to stand up for myself and not let anyone walk all over me, this rule seemed absurd.  When she spoke to me, I had to struggle to empathize with her worldview.

15

D4 Overgeneralization of similarities

I first told her that I had a similar experience when I lost my brother and that her feelings were normal for someone who was grieving. Although we had not lived near each other, we both lived in towns with state universities and civilization and talked about the lack of stores and things to do in our small town and how we both expected there to be more here. I felt that she really appreciated talking to someone who understood what it was like to move here from another far way region of the country.

D5 Stereotypes

I began wondering if H would be uncomfortable with such an exchange. I made immediate assumptions that she would be because in traditional Korean culture I assumed that children would not be expected to make eye contact. I assumed that eye contact in a disciplinary situation may actually be disrespectful. I found myself siding with M’s mother by trying to rationalize her reasons for arranging for him to change schools. I was thinking as many people do about ‘black schools.” I was under the impression that M would not be able to survive the lure of gangs and drugs at the ‘black school.” But, automatically, I was ignoring the fact that all these things can happen at any school. I had the perception that a “white school” does not face such problems.

16

InterventionsInt1 ESL TutoringH has been talking about some English phonemes with which she struggles. We made a list of words that include the phonetic sounds that she described, and we all practiced the sounds. We even talked about the tongue position and shape of the mouth for different sounds. We had a fun time and her pronunciation did improve!

Int2 Sharing objectsC suggested that I watch the movie "Never on Sunday" as a way of getting a better sense of Greek life.

Int3 Information exchange on customs and valuesShe told me how it is difficult for her to maintain the closeness of family that is expected in the Asian Indian culture. She feels that the American way of life puts more emphasis on work and independence. She struggles to find time for her family to spend together despite their hectic schedules. She also has noticed a difference in the way that each culture views public displays of affection.

17

Int4 Talking about family members, behaviors, practices

He spoke of how much he misses “this” part of Greek life and the lack of the concept of extended family in the American way of life. He talked about how people would just drop in if they were in the neighborhood. It would be an insult to pass by someone's house and not stop to say hello or visit. Int5 Client giving gifts

H had sent a special herbal tea home with me last week because my daughter had been sick. This week we brought H a little basketful of homemade cookies to thank her for the tea and for her hospitality. We all had a nice snack with cookies and tea.  Int6 Joking/humor

We did have a good chuckle about the irony of the United States monitoring other countries' elections when we can't even figure out how to do an accurate recount in our own.

18

Int7 Advocacy/Social Justice

This interaction caused me to consider more proactive ways that I can react to this discrimination, including advocating for cultural diversity awareness in my home town. Int8 Power Differential/equalizing power

I apologized again for forgetting to provide her with my information last week. I also explained that I felt badly because it was insensitive of me that the dynamic resulted in my ability to call her but her inability to call me. We talked about this power differential in concrete and simple terms and she confirmed that it had been a strange feeling for her.  Int9 Acculturation of client

Having been here seven years now, C feels that he is used to life in the US. On a recent visit home to Greece, he realized that, in some ways, he is more accustomed to life here than he is to some customs in his own country.

19

Int10 Racism experience of client

She often felt that no matter how hard she studied, her grades were lower in classes in which she had a White teacher. Her schools were integrated and she mentioned that many times the White teachers would be more inclined to call on White students. C felt that she performed better in classes in which her teacher was African American. Int11 Worldviews of client and counselor

H comes from a culture where guns are much less prevalent in the hands of the public; a culture where "the right to bear arms" is not a personal right. This discussion provided many opportunities for me to become very aware of the American ideals that I have inherited. I discovered that H’s worldview is very much based on trust issues with non‑minority individuals, whereas my worldview affords me the luxury of not being concerned about retribution for my opinions. It is an unreal experience to realize the amount of safety I feel in going about my daily life as opposed to the amount of insecurity many minority individuals must live with daily.  

20

Int12 Discussion of comfort levelWe discussed our comfort levels about the conversation we just had. Because I am a direct person, I asked A if she preferred a close or distant personal space comfort zone. She laughed and responded that it depended on who the person was and how well she knew her or him. Right after she answered my question I felt my cheeks blushing. 'What an asinine question I just asked her!' I thought to myself. I did not know what to say so I was quiet for a second. Then A asked me, 'Well, how about you? Do you like close or far personal space?' This made me smile because I had the same response that A did. Int13 Taking a chance/riskI took the chance to H directly if she was uncomfortable; if the exchange was too confrontational, etc. Much to my surprise she said that it was not and that she actually liked the approach. Int14 Asking questions/hypothesis testingI not only learned more about H's level of acculturation today, I also learned about the importance of asking questions and testing assumed knowledge. Had I not asked, I would have missed out on very important information.

21

Int15 Assuring confidentiality

I did assure her our discussions were held in confidence and that her identity was not disclosed in my project or to anyone else for that matter. She stated that she had been curious about the extent to which her identity was shared and that she was relieved that her personal information was considered confidential. I realized at this point that I had not done enough by way of introduction of this project to assure her of her privacy.  Int16 Setting goals

Today we talked about many things, such as politics, religion, fear, rural life, and adaptation. We discovered some areas that we would like to explore in more depth the next few times that we talk. Int17 Client tells a story

This was the first time her two sons were present during meeting. I noticed quite a difference in the way she interacted with each of them. I mentioned the difference that I was noticing to P and she started telling me about the time during which she was pregnant with S.

22

23

ANOVAs for Session Difference in Connection and in Disconnection

Sum of Square

sdf

Mean Square

F Sig.

MEAN_ Between

CONN Groups

Within

Groups

Total

5.086

54.361

59.477

9

187

196

.565

.291

1.944

.048

MEAN_ Between

DISCONN Groups

Within

Groups

Total

.302

7.760

8.062

9

187

196

.034

.041

.810

.608

Note. A Post Hoc HSD indicated significant difference between session 1 and session 9 (p<.01) on Connection