Assessing Pragmatic Understanding: A Description of One...

12
135 比治山大学現代文化学部紀要,第19号,2012 Bul. Hijiyama Univ. No.19, 2012 1. INTRODUCTION Since 2006 the author has taught an introductory course designed to foster pragmatic awareness in the belief that pragmatic awareness is essential for the development of communicative competence (Tanaka, 2006) and the enhancement of intercultural understanding and because most Japanese students of English have had little if any significant exposure to instruction in basic pragmatic concepts (Azuma, 1994). The course reported on here has focused on the instruction of eight common speech act sets (Ishihara & Cohen, 2010) and introduces students to relevant concepts in pragmatics and intercultural communication. The Heart to Heart textbook used (Yoshida et. al, 2000) was chosen in part because it is based on speech act data elicited from 150 American and Japanese university students, allowing for insights into the influence of culture on language and on the production of speech acts. An overview of this course is provided in Mosher (2010). For another useful overview, from one of the authors of this textbook, see Kondo (2008). This article focuses on the assessment of pragmatic understanding, which is a new area in second language testing for which there are as of yet no commonly accepted methods or measures (Hudson, Detmer & Brown, 1995; Brown, 2001; Ishihara & Cohen, 2010). One reason is that unlike other areas of language, such as grammar, listening or reading comprehension, there is usually no single correct answer. Furthermore, what is an appropriate answer in one situation, may be totally inappropriate in another situation, or presenting even more difficulties for pragmatic test makers, an expression that conveys a pragmatically appropriate meaning when uttered in one tone of voice, may be totally inappropriate in a slightly different tone of voice (Ishihara & Cohen, 2010), making the written assessment of pragmatic competence difficult. Here, the focus is limited to a brief description of the type of written assessment items developed for this course, fully realizing that alternative assessments, such as small group presentations, task sheet homework, office hour and classroom discussions are also valuable tools for assessing pragmatic understanding. First, a brief overview of the textbook and supplementary materials is provided. Next, the written exam item types used to assess students’ comprehension of the course content are described. Finally, the article concludes with a brief discussion of issues for further improvement. Assessing Pragmatic Understanding: A Description of One Classroom Approach David M. Mosher

Transcript of Assessing Pragmatic Understanding: A Description of One...

Page 1: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

135比治山大学現代文化学部紀要,第19号,2012Bul. Hijiyama Univ. No.19, 2012

1. INTRODUCTION

Since 2006 the author has taught an introductory course designed to foster pragmatic

awareness in the belief that pragmatic awareness is essential for the development of communicative

competence (Tanaka, 2006) and the enhancement of intercultural understanding and because most

Japanese students of English have had little if any significant exposure to instruction in basic

pragmatic concepts (Azuma, 1994).

The course reported on here has focused on the instruction of eight common speech act sets

(Ishihara & Cohen, 2010) and introduces students to relevant concepts in pragmatics and

intercultural communication. The Heart to Heart textbook used (Yoshida et. al, 2000) was chosen in

part because it is based on speech act data elicited from 150 American and Japanese university

students, allowing for insights into the influence of culture on language and on the production of

speech acts. An overview of this course is provided in Mosher (2010). For another useful overview,

from one of the authors of this textbook, see Kondo (2008).

This article focuses on the assessment of pragmatic understanding, which is a new area in

second language testing for which there are as of yet no commonly accepted methods or measures

(Hudson, Detmer & Brown, 1995; Brown, 2001; Ishihara & Cohen, 2010). One reason is that unlike

other areas of language, such as grammar, listening or reading comprehension, there is usually no

single correct answer. Furthermore, what is an appropriate answer in one situation, may be totally

inappropriate in another situation, or presenting even more difficulties for pragmatic test makers,

an expression that conveys a pragmatically appropriate meaning when uttered in one tone of voice,

may be totally inappropriate in a slightly different tone of voice (Ishihara & Cohen, 2010), making

the written assessment of pragmatic competence difficult.

Here, the focus is limited to a brief description of the type of written assessment items

developed for this course, fully realizing that alternative assessments, such as small group

presentations, task sheet homework, office hour and classroom discussions are also valuable tools for

assessing pragmatic understanding. First, a brief overview of the textbook and supplementary

materials is provided. Next, the written exam item types used to assess students’ comprehension of

the course content are described. Finally, the article concludes with a brief discussion of issues for

further improvement.

Assessing Pragmatic Understanding:A Description of One Classroom Approach

David M. Mosher

Page 2: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

2. TEXTBOOK OVERVIEW

2.1 Heart to Heart Textbook Contents

The textbook consists of twelve chapters. Eight of the chapters focus on the instruction of one

common speech act set. Every third chapter is a review chapter which provides additional speech act

examples from the previous two chapters in two short dialogs and gives students more opportunity to

role play the speech acts they have studied in different contexts. This article will focus on the non-

review chapters that form the core of the course.

The eight speech act sets covered are: compliments and responses to compliments, thanking,

requests, refusals, complaints, apologies, proposals (i.e., invitations) and disagreeing.

2.2 Speech Act Database

The speech act data consists of discourse completion task (DCT) results for the first speech act

elicitation situation of each of the eight chapters. The DCT were completed by 150 university students

in the United States and in Japan. Fifty students were native speakers of American English (A) living

in the United States. Fifty were Japanese university students who completed the DCT in English (JE),

their second language. And fifty were native speakers of Japanese (J) who completed the DCT in

Japanese.1 Both of the Japanese university student groups were residing in Japan, and members of

neither group had spent more than half a year studying abroad.

2.3 Major Activity Types

Each of the eight chapters, which introduce a new speech act set, is organized into six sections that

progress from activities designed to foster intuitive awareness of the act, to a study of the specific

speech act strategies (c.f., Ishihara & Cohen, 2010) found in the data for each of the participant

groups, to application of this new knowledge and awareness to role play activities, and finally to a

critical incident (c.f., Cushner & Brislin, 1996) that illustrates the kind of miscommunication speech

act differences can cause in intercultural communication. Each activity type is briefly described below.

―F―e―el ―th―e ―A―ct: The pedagogical goal of this activity type is to give students an intuitive feel for the

speech act to be studied by listening to two short dialogues. One dialogue reflects an American style

enactment of the target speech act and the other illustrates a Japanese style as determined by the DCT

data. After listening to both dialogues, students are asked which dialogue feels more American and

which feels more Japanese. Students can be asked which features of the dialogue influenced their

answer, and the instructor can write the dialogues on the board for closer examination of the

differences.

―D―o ―th―e ―A―ct: In this section, students read a short situation that calls for use of the target speech act,

then they are asked to complete a short dialog using the target speech act and role play it with a

classmate. At this point, students are not given any information on differences in American or

Japanese communication styles for this act since the goal is to activate students’ awareness of their

own communication styles before the act is studied in detail.

David M. Mosher136

1 These three groups are referred to hereafter as A, J and JE.

Page 3: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

―T―hi―n―k ―a―bo―ut―t―he―A―ct: Here, students study the linguistic means or types of expressions used by the

DCT participants to enact each speech act. For example, for refusals students learn that the DCT

participants used six types of expression or strategies for the situation given in “Do the Act. ”The six

strategies were the expression of positive opinions, thanking, apologizing, giving alternatives, making

direct refusals and providing reasons for refusing. Using this knowledge, students listen to short

dialogs that utilize each strategy type to see if they can identify the type. After this, they look back at

the dialogue they wrote to see which strategies they used in their own English for situation one (Do the

Act).

―C―r―os―s-―

C―ul―tu―ra―l ―C―o―m―m―u―ni―ca―ti―on――N―o―t―es: Students now examine a graph, which displays the numbers

of each speech act strategy type used by each of the three participant groups. They are asked to

compare each of the groups to find similarities and differences and to consider possible cultural and

linguistic reasons for the differences. Even students with very limited intercultural experience can be

encouraged to consider reasons for the native Japanese speaker (J) strategy choices and for the

potential effect of English as a second language on the JE participant group. If students’ English

abilities are low, this phase of the lesson is best done in Japanese.

―U―s―e ―W―h―a―t ―Y―o―u’―v―e ―L―ea―rn―ed: For this activity, students listen to and role play model dialogs before

completing their own short dialogs for two new situations. The goal now is for students to imitate the

American style for the target speech act so as not to cause any misunderstanding. The textbook

provides a short list of “useful expressions” to help the students. The author found, however, that the

students needed more guidance to be able to write appropriately complex dialogues. The task sheets

designed to facilitate this complexity are explained in section 2.4 below and in Mosher (2010).

―C―ul―tu―ra―l ―E―y―e-―

O―p―e―n―er: Each non-review chapter ends with a critical incident that illustrates a

potential speech act based intercultural misunderstanding. Causes for the misunderstanding can be

clearly related to communication style, cultural norms, and value differences, which may be

influencing the enactment of the targeted speech act. An example of one way to facilitate a deep

analysis of these critical incidents is given in Mosher (2010).

2.4 Task Sheets

Task sheets have been designed for each non-review chapter that give students more detailed

guidance for each dialogue completion task than is provided in the textbook. Students are told which

speech act strategy types and the number of types that they should use, in order to make their

dialogue pragmatically appropriate for each situation. Advice is sometimes linked to the list of

technical terms, such the abbreviated list in 2.5; thus, providing a broader conceptual framework for

this activity as well as facilitating class discussion and student presentations (see also Mosher, 2010). It

is left up to the student to decide the best sequence for the speech act types.

2.5 Pragmatic Technical Terms

Students are given a list of annotated technical terms compiled from work in pragmatics and

intercultural communication, in order to provide a broader and richer conceptual framework for the

course (c.f., Cutting, 2002; Dodd, 1995; Lustig & Koester, 1999, Tanaka, 2006). The terms are indexed

to the first chapters in Heart to Heart for which they are deemed relevant. These concepts are taught

explicitly in class in conjunction with specific speech act sets, homework feedback and student

Assessing Pragmatic Understanding: A Description of One Classroom Approach 137

Page 4: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

presentations. An abbreviated list of examples follows. For a more complete list see Mosher (2010).

2.5.1 Student Handout (Abbreviated)

Here is a list of technical terms (専門用語) from pragmatics and intercultural communication that

we will use in this course. These terms will help us to understand and talk about the cultural, linguistic

and communication style differences, which we will find in the Cross-cultural Notes data and in the

Cultural Eye-openers of our textbook, Heart to Heart. The chapter numbers are only guidelines: these

terms may be useful in other chapters as well.

speech act An action preformed by the use of a linguistic utterance (言語的発話).

発話行為.

vague Not clear. Vague words depend on the context to make their meaning

clear. 曖昧。Vagueness. (Ch. 2~)

ambiguous The state of having more than one possible meaning. 両義(のあるこ

と);多義性,曖昧さ。Ambiguity. (Ch. 2~)

indirect Avoiding saying something in a clear or obvious way. Not going or not

expressing something in a straight line. 控え目,間接的。Indirectness.

(Esp., Ch. 10)

direct Saying exactly what you mean in a way that nobody can pretend that

they do not understand. 率直的;直接的。Directness.

context The physical and linguistic environment of a linguistic message. コンテ

キスト,文脈。(Ch. 4~)

high-context communication A communication style in which most of the meaning is in the context

and not in the words themselves. Speakers must use contextual

information to guess what the speaker wants to say. (Ch. 2 “Why do

you always apologize,” Ch. 4 “Dr. Macintosh...” & Ch. 7 “You should’ve

told me”)

low-context communication A communication style in which most of the meaning is in the words

themselves. The listener does not need to guess (much). (Chs. 2, 4 & 7)

enthusiasm constraint Showing a lack of enthusiasm about something (熱意の欠如). This may

be interpreted negatively as meaning“no, ”for example, when the

person says“yes.”(Ch. 10 Eye-opener) (Tannen, 1984)

interjections A short sound, word or phrase spoken suddenly to express emotion or

enthusiasm. E.g., Oh! Wow! Ow! Oh-my-god! (SYN, exclamation)間

投詞;感嘆詞。

FTA Face threatening act. Speech acts that threaten someone’s face. 面目を脅

かす(発話)行為。FTA weight (W) = SD (S, H) + P (S, H) + R. (Ch. 4~)

face A person’s public self-image. One’s dignity (尊厳,自尊心). The desire

to be respected. The expectation that one’s public self-image will be

respected. 体面,面目,メンツ。(Ch. 4~)

positive face The need to be connected, to belong to a group. 積極的面目は「連帯」

(solidarity)の願望である。(Ch. 4~)

David M. Mosher138

Page 5: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

negative face The need to be independent, not imposed on by others. The right to

have freedom of action. This is also called independence face because it

is the psychological need for independence from others. 消極的/独立

的メンツは「独立」(independence)の願望である。(Ch. 4~)

negative politeness Showing awareness of another’s right not to be imposed on. (Ch. 10)消

極的丁寧さ,独立的丁寧さ。E.g., Could I use your pen? (Ch. 4~)

positive politeness Showing solidarity with another. 積極的丁寧さ,連帯丁寧さ。

E.g., Hey, how about letting me use your pen? (Ch. 4~)

social distance 社会的な距離(SD). The degree of closeness between the S (speaker)

and H (hearer). Age, status and degree of familiarity are three

important factors that determine SD. Examples from close to distant:

family (brother or sister), best friend, classmate, first time

acquaintance, a stranger. (Ch. 4~)

relative power 相対的権力(P). The relative degree of power that S or H have over

each other. For example, a boss has relatively more power than his/her

employee. A professor has relatively more power than his/her student.

(Ch. 4~)

ranking of imposition 押し付けのランク(R). The degree of imposition caused to S or H

because of the speak act. For example, how much will it inconvenience S

or H, or cause them to lose face (メンツを失う)? The R for a request

for the salt is much smaller than the R for a request for a letter of

recommendation. (Ch. 4~)

hesitation words Words like uhhh, ummm, etc. Hesitating is a way of softening a speech act.

softener Words and expressions that soften the impact of a speech act. Softeners

are often used in front of a speech act. 前置き表現は反対意見を聞きや

すくする役割を果す。(Ch. 4 & 7)

3. ASSESSMENT ITEM TYPES

Below are some examples of assessment items used in a 100-point written exam. The multiple-choice

items in section I, are designed to test students ability to identify the major differences between the

three groups that participated in the Heart to Heart DCT study. Section II, tests students’ ability to

recognize the strategy types used in each speech act set. Section III, tests comprehension of the

vocabulary in the pragmatic and technical terms list (c.f., section 2.5). This, of course, is not a direct

test of pragmatic comprehension. Next, the dialog items in section IV were drawn from students’

written task sheet homework (c.f., Mosher, 2010). Responses that were deemed inappropriate were

matched against responses that closely fit the textbook DCT data for the American university

students. Earlier versions of tests developed for this course used two or so written discourse completion

items instead, but such items take more time to grade and limit the number of items that can be

included in a test. Section V tests students’ ability to comprehend and analyze the critical incidents in

the textbook. Finally, it is important to note that this test is given as an open-book exam since the

overall purpose of the course is consciousness raising rather than memorization.

Assessing Pragmatic Understanding: A Description of One Classroom Approach 139

Page 6: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

3.1 Assessment Item Examples

I. C.C. Notes (20 points...2pts. each)

1. A use about twice as many compliment types as J or JE.

a. compliments b. no mentions c. thanking d. information

2. Which group uses the most casual request types?

a. American students b. Japanese students learning English c. Japanese students

II. Dialog Analysis (40 points...2pts. each)

―C―o―m―p―li―m―en―ts ―& ―R―es―po―ns―es. A: No Mention; B: Compliments; C: Question; D: Thanking; E: Information

A: Hi, Richard. C―I ―re―al―ly―li―k―e ―yo―u―r j―ac―k―et.

B: D―T―h―an―k―y―o―u.

―R―eq―u―e―s―ts. A: Very Polite Request: B: Polite Request; C: Casual Request

A: Hey, Midori, E―c―o―ul―d ―y―ou―p―a―s―s ―m―e ―th―e ―s―u―g―a―r?

B: Sure. Here you are.

III. Vocabulary Matching: Find the correct technical term for each of the following items, and write the

answer on your answer sheet. (10 points...1pt. each)

a. a vague word b. face c. negative politeness d. positive politeness e. context

f. softener j. social distance h. relative power i. hesitation word

6. The physical or linguistic environment of a linguistic message. (e)

7. A person’s public self-image: their dignity. (b)

8. Words like uhhh, ummm. (i)

9. Words and expressions used in front of a speech act to lessen its impact. (f)

10. Can I use your pen? (c)

11. Sumimasen. (a)

12. The degree of closeness between the speaker and the hearer. (j)

IV. Dialogs: Choose the most typical American style dialog line (台詞) for each of the following

situations. (10 points...1pt. each)

13. Situation: B bought a new pair of shoes.

a. A: Hey, those are nice shoes. Are they new? B: Yeah, I got them at...

b. A: I really like your new shoes. B: Thanks. They’re my favorite pair.

14. Situation: B invites you to go on a camping trip with his friends next weekend, but you can’t go.

a. A: I’m sorry, but I’m very busy next week. B. I’m really sorry.

b. A: Oh, I’d love to, but I really can’t B. I have a big math test on Monday.

V. Cultural Eye-Opener: Read the following eye-opener and answer the questions below.

20 points

David M. Mosher140

Page 7: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

"You should have told me!"

Kenji, a Japanese student studying in the U.S., shares a room in a dormitory with Mark. Mark, who

has a part-time job at night, usually comes back very late, around two o’clock in the morning. Every

night Kenji is bothered by the light Mark turns on when he enters their room and is having trouble

sleeping. However, Kenji was afraid to tell Mark about it, because he thought it would ruin their

friendship. So Kenji just said to him, “You were late last night,” hoping that Mark would realize the

situation. Kenji kept quiet for a month, but finally came to the point where he couldn’t take it anymore.

He said, “I’m leaving this dorm. I’m really angry, because you are such an insensitive person. Didn’t you

know that I was having difficulty sleeping because of you?” Mark had no idea what he was talking about

and said, “You should have told me. Since you didn’t say anything, I thought everything was fine!”

15. What’s the main problem? Explain in detail. Then, answer each of the following questions. If you

were in the same situation, would you find it difficult to complain, too? Would you do the same thing as

Kenji? Why or why not? (15 points)日本語で答えてもいい。

16. Rewrite the underlined dialog in English to make a good American style complaint for Kenji. (5

points)

3.2 Explanation of Assessment Types

A C. C. Notes

For these test items, only major patterns of differences in the textbook DCT data are assessed since

given the sample size, small differences would hardly be statistically significant nor realized in many

intercultural encounters. Moreover, attempting to memorize lots of minutiae for each speech act set

would, need-less-to-say, be both very difficult and of low utility. Furthermore, it would risk turning

students off to the very study of pragmatics.

Example one above tests students’ awareness of the major difference between the American and

Japanese students’ compliments. Namely, the American informants saw compliments as an

opportunity to exchange information about the object of the compliments via questions and answers. In

example two, the Japanese students learning English (JE), and doing the DCT in English, stand out in

their use of causal requests to the dean of their college for a letter of recommendation. Among fifty JE

students, there are 35 tokens of casual requests whereas, the fifty Americans use only seven and the

Japanese informants use none. In this situation, both of the later two groups show an overwhelming

preference for the most polite type of request of the three politeness levels identified in the data.

B Dialog Analysis

These items are undoubtedly the easiest of the five item types and are taken directly from example

dialogs in the textbook. The goal is to test recognition of pragmatic meaning in easy everyday

conversation. The comparatively large number of items require both some fluency in reading and some

speed of analysis.

C Vocabulary Matching

This item type is a straightforward attempt to assess students’ understanding of the pragmatic and

Assessing Pragmatic Understanding: A Description of One Classroom Approach 141

Page 8: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

intercultural communication concepts taught in the course. Class discussions, task sheet homework and

small group presentations are alternative more holistic means of assessing students’ grasp of these

concepts and their ability to apply them to real world problems like those in the Eye-Openers.

D Dialogs

The correct answer to item 13 is “a” since the DCT data shows that American students do far more

information exchanges (i.e., questions and answers) in their compliments than either group of

Japanese students. Distractor “b” was taken from student homework. The correct answer to item 14 is

“b.” In the refusal data, Americans use more direct refusals (“I really can’t”) than J, but they preface

them with positive opinions (“I’d love to”) or thanking types considerably more often than either

Japanese group. Distractor “a” was, in this case, taken from a textbook dialog more reflective of a

typical J or JE refusal which incorporate more apology types than A do. This item type is another way

to assess students grasp of the major differences between A, J and JE across a range of speech act sets.

As in item type A above, only major differences are singled out for assessment.

E Cultural Eye-Opener.

To answer the Eye-Opener question students have their classmates’ group presentation handout,

such as the following, available to them. This includes a Japanese translation of the critical incident and

the presentation groups’ bilingual pragmatic and cultural analysis of the situation. In addition to the

presenters’ analyses there are instructor comments designed to remind students of additional or more

in-depth explanations for the misunderstanding. Often, these teacher comments include terms from

the technical terms handout (2.5.1) that are particularly relevant to the incident.

Group Presentation Handout Excerpt with Eye-Opener Translation and Analysis

〈日本語訳〉 「あなたは私に伝えるべきです!」

ケンジは日本人の生徒でアメリカに留学しています。そして,マークとルームシェアをしています。

マークは深夜のアルバイトをしていて,いつも深夜2時くらいに帰ってきます。毎晩ケンジはマーク

がバイトから帰って部屋に入ってくる時の電気の光に悩まされ,睡眠不足になっています。しかしケ

ンジは,そのことをマークに怖くて言えません。なぜなら,ケンジは2人の友情が壊れると思ったか

らです。そこでケンジはただ「昨日遅かったね。」とだけ言って,マークが気づいてくれるのを待っ

ていましたが,マークが気づくことはありませんでした。ケンジは1カ月だけ何も言わず黙っていま

した。しかし,これ以上我慢できなくなったケンジは「この部屋を出る。君の鈍感さにはもう我慢で

きない。君のせいで夜眠れなかったのが分からなかったの?」とマークに言いました。マークは何の

ことだかさっぱり分からず,「それならそうと言ってくれれば良かったのに。何も言わないから何も

問題ないと思っていた。」とマークは言いました。

1. What’s the main problem? 主な問題点は何か?

Although Kenji complained indirectly and hoped Mark would notice, he did not notice.

ケンジは気づいて欲しくて,間接的に文句を言ったが,マークは気づいていなかった。

2. If you were in the same situation, would you find it difficult to complain, too?

もしも同じ状況だったら,あなたも不満を言うことができますか?

David M. Mosher142

Page 9: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

I would complain because if an indirect complaint does not communicate what I want, the only way

is to more directly communicate what is bothering me.

言います。なぜなら,間接的に言っても伝わっていないのだから,はっきり言ったほうが確実に

困っていることが伝わるから。

Items1and2of the handout are directly relevant to test item question number 15 above. The

students’ response to the first question identifies the basic linguistic problem, but it fails to relate it to

broader notions of communication style differences. The first instructor comment at the end draws

students’ attention to the clash between Kenji’s high-context communication style and Mark’s low-

context communication style. The second comment attempts to make the nature of Kenji’s weak hint

more explicit to students by highlighting what he does not say, so they can see how much is left up to

Mark to infer-too much, it turns out, for a low-context communicator like Mark. It could also be

pointed out here, that from Mark’s perspective, Kenji is not being cooperative enough (Tanaka, 2006 ;

Koizumi, 2001; Yule, 1996; Grice, 1975).

3. Write an acceptable American style complaint for Kenji.

ケンジの為にアメリカンスタイルで好ましい事を書いてください。

I’m having a little trouble getting to sleep. Would you use the small light when you come back

late from your part-time job?

眠れないから困っています。バイトから帰ってきたら,豆電にしてもらえますか?

Suggestion: Do you think you could just turn on the small light when...? 〈Instructor Comment〉

The student presenters’ response to this question is directly relevant to test item number 16 above.

Their complaint begins with a clarification of the situation that includes a softener or down grader (i.e.,

a little) and a specific request to Mark for a change in behavior that would help solve the problem. In

the textbook DCT data, American informants used clarifications and requests twice as often as either

the J or JE informants. The differences in softener strategy use were even more dramatic: A used

thirteen, whereas, J used only two and JE used none.

Instructor Comments

A Both Kenji and Tsutomu are also high-context communicators, but Mark and Dr. Macintosh are

low-context communicators.

B In this situation Kenji is very indirect! He only hints at the problem. He says nothing about the light

or about not being able to sleep. He expects Mark to “read the situation” and guess what the

problem is (空気を読んで問題を悟ること).

4. Conclusions

As noted, the assessment of pragmatic understanding is not any easy task. Here, however, the focus

has been not on an investigation of these difficulties, but rather on a description of one classroom

approach to written assessment of pragmatic understanding. Namely, a description of five item types

was provided: A multiple choice items designed to test awareness of major differences in the informant

Assessing Pragmatic Understanding: A Description of One Classroom Approach 143

Page 10: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

groups in the Cross-Cultural Notes DCT data; B multiple choice items designed to test recognition of

the communicative force/intent of dialogs excerpted from the text book; C vocabulary matching items

to check for receptive recognition of technical terms; D binary multiple choice questions to assess

ability to recognize the main American informant style shown in the textbook data versus a more

Japanese style usually taken from student homework; E open-ended questions regarding typical

incidents of cross-cultural misunderstanding from a pragmatic, cultural and intercultural

communication perspective.

As noted in Mosher (2010), the instruction of pragmatics opens students up to a whole new level of

linguistic meaning (Thomas, 1995). Although they are at least tacitly aware of the pragmatic force of

their utterances in their first language, they often have little or no awareness of the communicative

intent of even very basic everyday English conversation. The Heart to Heart textbook helps enhance

students’ awareness of this pragmatic level of meaning systematically. As they progress through the

book, they can see that several speech act strategies are used in more than one speech act set. For

example, compliments are used not only in the compliment and response speech act set of the first

chapter, but as one thanking strategy in chapter two. Thanking which is examined in chapter two is

also a strategy type used in compliment responses as well as in refusals. Explanation and information

strategy types appear in the complement, refusal, complaint, apology and disagreement speech act set

data. Cumulatively, this gives students a fairly rich picture of the use of speech acts in English vis-a-

vis Japanese.

Each time the course has been taught, certain themes have emerged. One that is particularly

noteworthy is that detailed information and explanations seem more important for Americans than for

Japanese in a number of speech act situations. For example, when students complete task sheets and

presentation handouts for compliments and responses (chapter2), refusals (chapter5), complaints

(chapter7), apologies (chapter8) or disagreements (chapter 10), their explanations or reasons for

their disagreement are almost invariably much more succinct and less detailed than seem sufficient to

native speakers, illustrating quite vividly over the length of the course the relatively high-context

communication style of Japanese speakers and the low-context communication style of Americans.

This not only provides multiple teachable moments for the instructor, but also suggests the most

relevant and appropriate areas for assessment.

In conclusion, I will briefly mention a few of what seem to be the more promising ideas for the

improvement of pragmatic assessment. First, items can to be designed to more clearly focus on either

comprehension or on production (Ishihara & Cohen, 2010). The development of discourse completion

assessment items is a well-researched approach for assessing productive ability that bears promise.

Hudson, et al. (1995) provide a particularly nice inventory of such item types. Second, the use of

translation to both heighten pragmatic comprehension and to assess understanding of pragmatic

meaning may provide a useful assessment tool for Japanese students (House, 2008). Similarly, testing

students in Japanese may yield more valid results as Brown (2001) notes. Third, the conscious

incorporation of direct and indirect means of assessing comprehension and production bears merit

(Ishihara & Cohen, 2010). For example, indirect means of assessment might provide interesting ways

to assess students’ awareness of the effect of the cooperative principle (Tanaka, 2006; Koizumi, 2001;

Yule, 1996; Grice, 1975), such as, A’s need for more detailed explanations across a range of speech act

sets. Finally, the availability of Brown and Levinson’s (1987) seminal book on politeness in Japanese

David M. Mosher144

Page 11: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

provides interesting opportunities for the teaching and assessment of politeness (Brown & Levinson,

2011) and raising students’ awareness of its role in building rapport (Spencer-Oatly, 2004). The

teaching of politeness can be augmented with resources like Vardeaman’s (2000) and on-line corpses

(Ishihara & Cohen, 2010) to provide students with a richer range of English expressions for each

speech act set from which to choose and study.

References

Azuma, S. (1994). Teinena Eigo-Shitsureina Eigo: Eigo no Poraitonesu-Sutoratejii [Politeness

Strategies]. Tokyo: Kenkyusha.

Brown, J. D. (2001). Pragmatic Tests: Different Purposes, Different Tests. In K. R. Rose & G. Kasper

(eds.), Pragmatics in Language Teaching. Edinburgh: Cambridge University Press.

Brown, P. & Levinson S. C. (1987). Politeness: Some Universals in Language Usage. Cambridge:

Cambridge University Press.

Brown, P. & Levinson S. C. (2011). Poraitonesu: Gengo Shiyou ni okeru, aru fuhen genshou [Politeness:

Some Universals in Language Usage] (N. Tanaka, Trans.). Tokyo: Kenkyusha.

Cutting, J. (2002). Pragmatics and Discourse: A Resource Book for Students. London: Routledge.

Cushner, K. & Brislin R. W. (1996). Intercultural Interactions: A Practical Guide, 2nd ed. Thousand

Oaks, CA: Sage Publications.

Dodd, C. H. (1995). Dynamics of Intercultural Communication, 4th ed. Madison, Wisconsin: Brown and

Benchmark Publishers.

Grice, P. (1975). Logic and Conversation. In Cole, P. & Morgan J. L. (eds.), Syntax and Semantics

Volume3: Speech Acts. Academic Press.

House, J. (2008). Using Translation to Improve Pragmatic Competence, pp. 135-152. In Soler, E. A. &

Martinez-Flor, A. (Eds.), Investigating Pragmatics in Foreign Language Learning and Testing.

Bristol, UK: Multilingual Matters.

Hudson, T., Detmer, E. & Brown, J. D. (1995). Developing Prototypic Measures of Cross-Cultural

Pragmatics. Honolulu: University of Hawai’i Press.

Ishihara, N. & Cohen, A. D. (2010). Teaching and Learning Pragmatics: Where Language and Culture

Meet. Edinburgh: Pearson Education Limited.

Koizumi, T. (2001). Nyuumon Goyouron Kenkyuu: Riron to Ouyou [An Introduction to Pragmatics

Research: Theory and Application]. Tokyo: Kenkyusha.

Kondo, S. (2008). Pragmatic Development Through Awareness-raising Instruction, pp. 153-177. In

Soler, E. A. & Martinez-Flor, A. (Eds.), Investigating Pragmatics in Foreign Language Learning

and Testing. Bristol, UK: Multilingual Matters.

Lustig M. W. & Koester, J. (1999). Intercultural Competence: Interpersonal Communication Across

Cultures, 3rd ed. New York: Addison Wesley Longman.

Mosher, D. (2010). Fostering Heart to Heart Communication Skills, Ver. 1. The Bulletin of Hijiyama

University, 17, pp. 47-61.

Spencer-Oatey, H. (2004). Iibunka Rikai no Goyouron: Riron to Jissen [The Pragmatics of

Understanding Other Cultures : Theory and Practice] (R. Asaba, N. Tanaka, T. Tsurusaki, Y.

Tsuruda, M. Kumano & S. Fukushima, Trans.). Tokyo: Kenkyusha.

Tanaka, N. (2006). Puragumateikusu-Waakushoppu: Minomawari no Kotoba wo Goyouronteki ni Miru

Assessing Pragmatic Understanding: A Description of One Classroom Approach 145

Page 12: Assessing Pragmatic Understanding: A Description of One ...harp.lib.hiroshima-u.ac.jp/hijiyama-u/file/12031/20140306164236/12... · The eight speech act sets covered are: compliments

[Pragmatics Workshop: Looking at Everyday Language from a Pragmatic Perspective].

Yokohama: Shumpusha.

Tannen, D. (1984). Conversational Style: Analyzing Talk Among Friends. Norwood, NJ: Ablex

Publishing Corporation.

Thomas, J. (1995). Meaning in Interaction. London: Longman.

Vardaman, Jr., J. M. (2000). Yosoyuki Eigo, Fudangi Eigo [A Guide to Politeness in Spoken English]

(A. Tsukada, Trans.). Tokyo: Macmillian Language House.

Yoshida, K. et al. (2000). Heart to Heart: Overcoming Barriers in Cross-Cultural Communication.

Tokyo: Macmillian Languagehouse.

Yule, G. (1996). Pragmatics. Oxford: Oxford University Press.

KEYWORDS: pragmatic assessment, speech acts, communication styles

David M. Mosher(言語文化学科国際コミュニケーションコース)

(2012. 11. 15 受理)

David M. Mosher146