Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype...

36
Stefan Diemer [email protected] Saarland University and Trier University of Applied Sciences Paralanguage and gesture in a corpus of Skype conversations: “... okay so good luck with that ((laughing))?”

Transcript of Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype...

Page 1: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

[email protected] TrierUniversityof AppliedSciences

ParalanguageandgestureinacorpusofSkypeconversations:“...okaysogoodluckwiththat((laughing))?”

Page 2: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

• 200+hoursofSkypeconversationsbetweenspeakersofEnglishasaLinguaFranca,currentlybeingcompiledatSaarlandUniversity,Germany

• PartnersfromBulgaria,Spain,Italy,Sweden,Finland,France,Belgium,andL1referencecomponentsfromtheUKandtheUS(planned)

• Topicprompts,academiccontext,30-60min.• Allowsresearchintheareasof:

o EnglishasaLinguaFranca(ELF)o Lexicalinnovationo Pragmatics&DiscourseAnalysiso Computer-mediatedcommunication(CMC)o ...

CASE:CorpusofAcademicSpokenEnglish(forthcoming)

CASEwebsite:tinyurl.com/CASEproject

Page 3: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Researchbackground:Videocommunication

https://pixabay.com/de/kreis-rund-gew%C3%B6lbt-menschen-kreise-291098/

● Research on Skype as oral, synchronous and dialogic discourse typenot very extensive

● Research on video-conferencing as a predecessor of Skype:○ Videoandaudiochannelsimitatingfeaturesof

face-to-faceinteractions,facilitatinginformalcommunication(Fishetal.1993:50f.)

○ Additionalbenefits ofanaudio-visualvideoconnection:visualbackchannels,non-verbalresponsesandnon-verbalresponseforecasting,gesture-basedsupportofdescriptions,non-verbalexpressionofattitudes,betterpausemanagement(IsaacsandTang1994)

Page 4: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Researchbackground:Videocommunication

https://pixabay.com/de/kreis-rund-gew%C3%B6lbt-menschen-kreise-291098/

● Research on video-conferencing as a predecessor of Skype:○ Contributingtoa“markedlyricher,subtler,andeasier[communication]

than[…]telephoneinteractions”(IsaacsandTang1994:67)○ CMCfacilitatesinterculturalcommunicationregardlessofnational

boundaries(Gong2005:1)○ But“distancestillmatters” (OlsonandOlson2000:173)- issueswith

theaudiochannelsuchas“delays,echoes,[and]incomprehensibleaudioquality”poseasubstantial threat tosuccessfulinterpersonalinteractions(TangandIsaacs1993:193f.)

Page 5: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

● ELF“orientstoachievingmutualcomprehension”betweenspeakersofdifferentlanguageandculturalbackgrounds(Mauranen 2012:7)

● InELF“differencesfromnativeEnglishthatachievethis[successfulinterculturalcommunication][are]regardednotasdeficienciesbutasevidenceoflinguisticadaptabilityandcreativity”(Jenkins2015:45)

● ELFasasetofstrategiesaimedatachievingmutualandsituatedcomprehension(Mauranen 2012)

Researchbackground:EnglishasaLinguaFranca(ELF)

https://pixabay.com/de/kreis-rund-gew%C3%B6lbt-menschen-kreise-291098/

Page 6: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

BabyCASE775mins(13hours)ofSkypeconversationsfromCASE(ca.115000wordsintheannotatedversion)

Composition:● 5conversationseachfromCASESB-SF/FL/ST/HE● 10hoursvideo,3hoursaudio,

duration:30-45minutes,varioustopics● Averageage:ca.23years,70%female,30%male● AveragetimelearningEnglish:12.4years

Subcorpusforquantitativeanalysis:BabyCASE(forthcoming)

https://www.baby-connect.com/images/baby2.gif

Page 7: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

01SB01SF01:SB01: Icanhearyou,SF01: o:h great,

...atlast._((thh))SB01: yeah((laughing))that'strue,

Paralanguage

Rapportmanagement:LaughterindelicatesituationsLaughterasamethodforsuccessfulrapportmanagementinCASE(cf.Schmidt2015)

● Rapportmanagement○ Generalfeatures(Spencer-Oatey 2002,2005):politeness&laughter○ “Harmonybetweenpeople”(Spencer-Oatey 2000:16)○ Positiveeffectofinteraction;bonding(Provine 2000)○ Secondaryeffect:Tensionreleaseandsignalling relief

(Gregory1924,Grotjahn 1957,Meierkord 2000)

● InaCMCsituation○ Increasedsituationaldelicacy○ Novelty

● Createafeelingofnon-seriousness(≠humorous)

https://pixabay.com/en/girl-small-child-child-laugh-happy-1511690/

Page 8: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

01SB01SF01:SB01: Icanhearyou,SF01: o:h great,

...atlast._((thh))SB01: yeah((laughing))that'strue,

Paralanguage

Rapportmanagement:LaughterindelicatesituationsApartfromhumorouscontexts,laughterisalsousedasamethodforsuccessfulrapportmanagementinCASE(cf.Brunner,DiemerandSchmidtforthcoming)

● InaCMCsituation:Increasedsituationaldelicacy/novelty

● Laughtercontributestocreatingrapportbetweeninterlocutors(Spencer-Oatey 2002)bycreatingcommongroundandreducingthesituationalawkwardness(cf.Chafe2007)

● Asameansofmitigatingadelicatesituation,forexampleembarrassment,byindicatingnon-seriousness(Chafe2007),whereapotentiallyproblematicsituationcanbedefusedbymeansoflaughter(Jeffersonetal.1987:172)

https://pixabay.com/en/girl-small-child-child-laugh-happy-1511690/

Page 9: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

((LAUGHS)) separate,loudlaughter

((laughs)) separatelaughter

((laughing)) laughingwhilespeaking

((chuckles)) chuckle

((chuckling)) chucklingwhilespeaking

((thh)) aspiratedminimumlaughterstartingwithalveolarplosivesound

((ehh)) monosyllabiclaughpulse

((hehe)) shortchuckling

((heh)) initiallyaspiratedmonosyllabiclaughpulse

Transcription:Laughter

https://pixabay.com/en/girl-small-child-child-laugh-happy-1511690/

Page 10: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

07SB49FL33:SB49:u:hm,

Ithink it's likeu:hm…like,yeah typical stereotypeSauerkraut((German(1.2))),_((ehh))

FL33: ah,okay.€€€

SB49:uhm,Knödel((German(0.8))),..Idon't know if there is aword inEnglishfor that?[((chuckles))]

FL33: [((LAUGHS))](...)FL33: I'm sorryfor this conversation likehalfanhour talking about food,

and Ihaven't eaten yet.SB49: ((ehh))FL33: ((ehh))SB49: Ithink we don't have to talk the whole timeabout food,

[it's just],FL33: [no no].SB49: to start the conversation ((laughing)).FL33: yeah of course butstillI'm likemhm I'm hungry ((laughing)).

https://pixabay.com/en/girl-small-child-child-laugh-happy-1511690/

Example:Laughter

Page 11: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Casestudy:Paralinguisticbehavior- Laughter

Laughterasakeyelementininteraction

SB17: andIalwaysfindniceplaceswhereIthinkohI-onedayIwanna g-Iwouldliketogotheretoo,and..ifI'mnotstudentanymoreIcanaffordabit[more],

SF10: [yeah].SB17: ((heh))..yeah.

that'salways..uhm,[yeahyoualways]-

SF10: [soif]Ineeda..avacation,I:..know..((heh)) ..thatIcanturntoyou ((chuckling)).((chuckles))

SB17: ((heh))you- youcouldyesyoucould((chuckling)).ifyou[don'thave]((chuckling)),

SF10: [that’s()],((laughs))SB17: ifyoudon'thaveu:h officesin- ininBulgaria?

Example“Travelagency”(CASE07SB17SF10)

Page 12: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Casestudy:Paralinguisticbehavior- Laughter

LaughterasakeyelementininteractionSB24: ..peoplecancometome.

andIlikehavetogiveonelessonaweek,Ishouldtalksomething?likehowthey...shouldrea:d theirmaterial,and..whattheyshouldreadandthingslikethat,Idon'tknow.

SF15: ..ohmygod,you'resocool.((laughs))

SB24: ..I'mnot,I'mjustworkingfortheprofessor.((laughs))...

SF15: (1.0)well:<y- youareImean>whogetstodothat,Idon'tgettodoit.

SB24: ...<I'm>sureyou'regoingtogettodoit.ifyoualreadystartedofflikethat,((laughs))

(CASE11SB24SF15)

Page 13: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

● Severalfilteringpossibilities● Preservationoffeaturesrelevantforlinguisticresearch● Gestures,stance,facialexpressions,materialobjects

Gestureinvideo:Anonymisation

Page 14: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

● Multimodalexplanationsandgestures(e.g.backchannels,greeting,leave-taking)

● Turn-takingthroughmulti-modalmeans(e.g.facialexpressions,gestures,stance)

● Third-partyinterferenceorparticipation

● Objectsastopicfocus

Gestureinvideodata

Page 15: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

“Books”

Transcribingspokendata:Discussion

Page 16: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

“Books”(CASE03SB27FL25)

Transcribingspokendata:Discussion

1 SB27: I have so much books here that I .. bought,2 but .. I can’t read them. ((hehe))3 FL25: look, {moves camera around to “point” at the bookshelf and then extends her hand to point}4 I mean .. we have dictionaries,5 yes, dictionaries,6 but ther- but there are also books there somewhere, {waves hand to indicate “further away”}7 uhm?8 and I have SO many books to read {grasps head with both hands} it’s just,9 .. SO: FRUstrating,10 because I like to read them all at the same time,11 and it´s […] so frustrating, {makes fists}12 a:h.13 SB 27: ((ehh))

Page 17: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

“Books”

Issues:● Gestures(pointing,emphasis,turn-taking)● Facialexpressions● Laughter● Physicalstance

● Multimodalexplanations

● Cameraperspective

● Objectfocus

● ...

Transcribingspokendata:Discussion

Page 18: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

“Pretzels”:Audioanalysis1ST07: [so]°ho-°,2my..m:,3Iwouldliketoknow,4..howisa,5..Germanpretzel?((ehh))6SB57: (1.3)uhm so,7 (1.4)uh[m:],8ST07: [forins]tance,9 ..ho- ..howitismade?10...isituh..uh..howcanIsay,isituhm ...more

likeabiscuit?11...isitmorelikeabread?12SB57:(2.2)it'suh..hm ..hm,13I'dsayit'sactuallymorelike..m:,14(2.0)<uh>...hm,15(1.5)also((German(0.3)))..uhm uh,16..uhm Iknowthat,17uhm:pretzelsaremadeby,18..beingdippedinto..kindofanacid?

19 ST07: ... mhm=20 SB57: =that's why they have this,21 .. brown .. crust things,22 ST07: ... yes,23 SB57: .. and there are salt on them,24 ST07: ... mhm,25 SB57: .. and they are usually .. like (2.0),26 .. they have a knot in them,27 like the- .. they have [this eight, eight] kind of shape.28 ST07: [okay okay]29 SB57: .. so,30 ST07: [m:],31 SB57: [are] are these the pretzels you have too?

Example “Pretzels” (CASE 07SB57ST07)

TheroleofmultimodalfeaturesinELFconversations:Casestudy

Page 19: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

“Pretzels”:Videotranscript

TheroleofmultimodalfeaturesinELFconversations:Casestudy

1ST07:[so]°ho-°,2my..m:,3Iwouldliketoknow,4..howisa,5..Germanpretzel?((ehh))6SB57:(1.3)uhm so,{raisinghandswithpalmsfacingup}7(1.4)uh[m:],8ST07:[forins]tance,9..ho- ..howitismade?10...isituh..uh..howcanIsay,isituhm ...more

likeabiscuit?11...isitmorelikeabread?12SB57:(2.2)it'suh..hm ..hm,13I'dsayit'sactuallymorelike..m:,14(2.0)<uh>...hm,15(1.5)also((German(0.3)))..uhm uh,16..uhm Iknowthat,17uhm:pretzelsaremadeby,18..beingdippedinto..kindofanacid?{imitatesdipping}

19 ST07: ... mhm=20 SB57: =that's why they have this,21 .. brown .. crust things, {outlines crust of a pretzel}22 ST07: ... yes,23 SB57: .. and there are salt on them, {imitates sprinkling salt}24 ST07: ... mhm, {nodding}25 SB57: .. and they are usually .. like (2.0), {showing the shape ofa pretzel with his index fingers and performing a repeatedtying motion}26 .. they have a knot in them,27 like the- .. they have [this eight, eight] kind of shape. {usesindex fingers to indicate the shape}28 ST07: [okay okay]29 SB57: .. so,30 ST07: [m:],31 SB57: [are] are these the pretzels you have too?

Example “Pretzels” (CASE 07SB57ST07)

Page 20: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

GesturesinBabyCASE

Shaking head

HE03:okaysoisitareyouactuallyrecordingatthemoment?SB106:yeah.HE03:okay.

yeah[that'sfine].SB106:[Istarted]right,

...withthecallbut,yeah,Icandeletethebeginningso,

HE03:noit'sfineIdon'tIdon'tmind{shakinghead}.[it's],

SB106:[okay].((laughs))HE03: ganz egal ((German(1.5))),

..für mich ((German(1.2)).SB106:ohyoucanspeakGerman.HE03: yeahalittlebit,

(CASE07SB50FL34)

Page 21: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

GesturesinBabyCASE

FL34: (1.0)interpretingislikethirtyfourPLAces ..andtranslationissixtyfour...soit'slike..HA:LF..thepeoplethatGETinside..th- ..[the:..u:h],

SB50: [ohokay].FL34: ..firstdegree.SB50: mhm.FL34: andyoudo- youhavetocountpeoplefromoutsidefromotheruniversitie:s,

..andyouhavetocountpeoplewhodidn't,

..uh..PASS..theTESTS..the:..yearsbeforeandthey'recrun- soit's-{shakinghead}(1.0)a:h fuckedupit'ss:o fuckingdifficult..[toget]inTHERE.

SB50: [mhm],FL34: (1.0)it's:..reallydifficult.#00:36:31-9#

andIknow- ..Iwill..try..both:..tests?

(CASE07SB50FL34)

Shaking head

Page 22: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

GesturesinBabyCASE

HE01: .."oh"?andwelloneandahalfyearisn’ttoobad.

SB75: ..<yeah>that’sokay, #00:16:30-7#..Icanseetheendalready?{pointingtothesideandlookingthere}<so>,

HE01: [((laughs))]SB75: [((laughs))]soitsreachable.

so.h.HE01: yeah.twellLondonisa..{shakinghead} isagreatcityI- Iah,

Iwanttomovethere((thh)),orlivethere.

SB75: ..ohreally,youALso.

HE01: .."yeah".SB75: it'samazingwecanhaveacoffeeTHEREthen.

[ifwearebothlivingthere?((hehe))]

(CASE01SB75HE01)

Shaking head

Page 23: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Quantitativeanalysis:GesturesinBabyCASE

764 instances of transcribed non-verbal behaviour in the preliminary transcripts

● Nodding (283 instances in BabyCASE) most frequently co-occurs with “yeah” (110), “mhm” (47),“right” (16), and “okay” (17) (within three preceding and following words), i.e. in more than twothirds of the contexts (67%) indicating (and emphasizing) support and agreement

● Shaking head (45 instances) does not only indicate (and underline) negation (co-occurring with “no”,verbal negation, “not”, also with questions, in 32 instances), but also despair (“it’s so fucking difficultto get in there”); awe (“London is a great city”); uncertainty (“in England or something like that”);self-correction (“for getting a teacher [...] (1.7) for becoming a teacher?”); lack ofalternatives/resignation (“everybody else just has to deal with some chocolate” as a gift); signalinglack of understanding (“what what one?”); andword search

Page 24: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Quantitativeanalysis:GesturesinBabyCASE

● Shrugging (24) is used, for example, in cases of uncertainty (“I don’t know why”); to indicatenormalcy (“I think it’s pretty much, like any other country in the world nowadays?”); to mark a lackof knowledge (“I’m missing the word?”); lack of alternatives/resignation (“that’s what it is”);acceptance (“{shrugs} right” as a response to the description of an unusual combination of subjects atuniversity); to indicate a lack of preferences (“you want to go first or should I? [...] Go ahead{shrugs}”); exasperation/frustration (“it doesn’t make sense, why is the table female?”);disapproval (“the topic is not, ((hehe)) {shrugs}”); embarrassment (“it’s kind of embarrassing toadmit, [...] but, {shrugs} yeah”); and for emphasis in combination with discourse markers (“yeswell {shrugs}”)

https://pixabay.com/de/panda-verwirrt-fragen-achselzucken-303949/

Page 25: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Quantitativeanalysis:GesturesinBabyCASE

● Pointing (13) is used to indicate objects in the physical surroundings; to refer to oneself or theinterlocutor; to signal general direction (“the north part of the city”); and for metaphoricalpointing (“I can see the end already”)

● Air quotes (10) are used when the English term is unknown (resulting in approximation, code-switching or paraphrase), but also for vague or unexplained terms (“Christmas stuff”);imprecise/generalizing terms (“Catholic” country); to distance oneself (Catholics “believe”); andto signal irony (being able to watch a lecture “again and again”)

https://pixabay.com/de/hand-finger-point-hinweis-1590381/

Page 26: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Quantitativeanalysis:GesturesinBabyCASE

● Mimickingactions (7)areusedtorepresentparticularactivities,suchasputtingeggsinabowl,limping,orwhipping,andtoindicateshapes,suchastheshapeofatortilla.

● Waving (6)isusedforleave-takinginBabyCASE,sometimesreciprocally

● Instancesofleaningforward (3),e.g.indicatingengagementatthebeginningoftheconversation(“Howareyou{leansforward}”),andleaningback (4),e.g.tosignalapossibleclosingpointoftheopening,oratopic(“that’scool{leansback}((clearsthroat))”).

https://pixabay.com/de/barack-obama-imitiert-mckayla-maroney-1174513/https://pixabay.com/de/kind-waving-goodbye-abreise-flugzeug-595429/

Page 27: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

● First insights into the interplay between gestures and spoken discourse● Point to tendencies regarding co-occurring words and, by extension, possible functions

of gestures in conversations● Need to be supplemented by a more detailed (qualitative) context analysis of the

individual occurrences to give a more comprehensive view of the dynamic nature ofcommunication and the meaning-making processes involved

→Gesturesaskeyelementsoflinguisticmeaning-making

https://pixabay.com/static/uploads/photo/2015/05/29/05/04/keys-788907__340.png

Quantitativeanalysis:GesturesinBabyCASE

Page 28: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Complexgestures

“Beer”(CASE03SB47FL21)

● Off-screencommunication

● Multimodalexplanation

● Objectsastopicfocus

● Physicalstance

https://pixabay.com/static/uploads/photo/2016/09/14/11/36/beer-1669275__340.png

Page 29: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Complexgestures

“Beer”:Analysis1 SB47: yeah we’re always in the kitchen drinking and [watching],2 FL21: [yeah ((laughing)) ()]3 {SB47’s roommates shout in the background}4 SB47: [OH..YEAH?]5 .. YEAH? {turns head to look at the roommates and nods}6 they are protesting but [it is reali]ty.7 FL21: [it’s true ((laughing)) ()],8 SB47: they are also like,9 always,10 like,11 uh,12 here. {holds a bottle of beer up with her right hand and points to it with her left index finger}13 Beer? {holds the bottle up to the screen}14 WE NEED BEER and,15 wine,((laughs)){holdsaglassofwineupwithherlefthandandraisesittothecamera,

andherroommate,off-screenclinksitwithherglass}16 EVERY time, ((laughs))

Page 30: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

ParalanguageandgesturesprovideadditionalcuestounderstandinginELFconversations• Laughterasahumordeviceandameansofmitigatingawkwardsituationsandcreatingrapport

• Gestures:o Backchanneling,turn-taking,greeting,leave-taking,stanceo Supportiveanddescriptivegestures,multimodalexplanationso Cameraperspectiveaspointingmechanism,objectshowingso Thirdpartyinterventionandoff-screencommunication

(cf.alsoBrunner,Diemer andSchmidtforthcoming)

Summary:TheroleofmultimodalfeaturesinELFconversations

Page 31: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson
Page 32: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Thank [email protected] TrierUniversityof AppliedSciences

CASE:tinyurl.com/CASEproject

Page 33: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Auer,P.(2001).Code-switching:Discoursemodels.In.R.Mesthrie(ed.),Conciseencyclopediaofsociolinguistics. Amsterdam&NewYork:Elsevier,443–446.Auer,P.(2005).Apostscript:Code-switchingandsocialidentity.JournalofPragmatics37,403-410.Auer,P.&Eastman,C.(2010).Code-switching.InJ.Jaspers,J.-O.Östman&J.Verschueren(eds.),SocietyandLanguageUse(HandbookofPragmatics7).Amsterdam:JohnBenjamins.Björkman,B.2009.FromCodetoDiscourseinSpokenELF.InA.Mauranen,A&E.Ranta(eds.),Englishasalinguafranca:studiesandfindings,225-251.Newcastle:CambridgeScholarsPress.Brunner,M.-L.,Burghardt,S.,Collet,C.,Diemer,S.&Schmidt,S.(Forthcoming). BabyCASE.Saarbrücken:SaarlandUniversity.Brunner,M.-L.,Diemer,S.&Schmidt,S..Forthcoming.“...okaysogoodluckwiththat((laughing))?”- Managingrichdataina corpusofSkypeconversations.StudiesinVariation,ContactsandChangeinEnglish. Helsinki:Varieng.Brunner,M.-L.,Diemer,S.&Schmidt,S.(2016)."It’salwaysdifferentwhenyoulooksomethingfromtheinside"- LinguisticinnovationinacorpusofELFSkypeconversations. InternationalJournalofLearnerCorpusResearch2016(2).CASE.Forthcoming.CorpusofAcademicSpokenEnglish.Saarbrücken:SaarlandUniversity.[http://www.uni-saarland.de/index.php?id=48492](25.09.2016).Chafe,W.1982.“IntegrationandInvolvementinSpeaking,Writing,andOralLiterature”.In:D.Tannen(ed.),SpokenandWritten Language:ExploringOralityandLiteracy,35-53. Norwood,NJ:Ablex.Chafe,W.1994.Discourse,Consciousness,andTime:TheFlowandDisplacementofConsciousExperienceinSpeakingandWriting.Chicago:TheUniversityofChicagoPress.Chafe,W.2007.TheImportanceofNotBeingEarnest:TheFeelingBehindLaughterandHumor. Amsterdam&Philadelphia:Benjamins.CLAWSpart-of-speechtaggerforEnglish.UCREL(UniversityCentreforComputerCorpusRe-searchonLanguage),LancasterUniversity.[http://www.comp.lancs.ac.uk/computing/research/ucrel/claws/](25.09.2016).Cogo,A.2009.AccommodatingdifferenceinELFconversation.eds.A.Mauranen&E.Ranta(eds.),Englishasalinguafranca:studiesandfindings. Newcastle:CambridgeScholarsPress,254–273.Cogo,A.&Dewey,M.(2006).EfficiencyinELFcommunication:Frompragmaticmotivestolexico-grammaticalinnovation.NordicJournalofEnglishStudies,5(2),59-93.

References

Page 34: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Cook,G.1995.Theoreticalissues:Transcribingtheuntranscribable.InG.Leech,G.Myers&J.Thomas(eds.),SpokenEnglishonComputer,35-53.Harlow:Longman.Diemer,S.,Brunner,M.-L.&Schmidt,S.2016.Compilingcomputer-mediatedspokenlanguagecorpora:Keyissuesandrecommendations.InternationalJournalofCorpusLinguistics21(3):349-373Dressler,R.A.&Kreuz,R.J.2000.Transcribingoraldiscourse:Asurveyandamodelsystem.DiscourseProcesses29(1):25-36.Edwards,J.A.1993.“Principlesandcontrastingsystemsofdiscoursetranscription”.InJ.A.Edwards&M.D.Lampert(eds.),TalkingData:Transcriptionandcodingindiscourseresearch, 3-32.Hillsdale,NJ:LawrenceErlbaumAssociates.ELFA.2008.TheCorpusofEnglishasaLinguaFrancainAcademicSettings.2008.Director:AnnaMauranen.UniversityofHelsinki.[http://www.helsinki.fi/elfa/elfacorpus] (25.09.2016).Fish,R.S.etal.1993.Videoasatechnologyforinformalcommunication.CommunicationsoftheACM36(1).48-61.Gee,M.2014.CASEXMLConversionTool. [http://rdues.bcu.ac.uk/case] (25.09.2016).Gfaranga,J.(2009).Code-switchingasaconversationalstrategy.Handbookofmultilingualismandmultilingualcommunication,ed.byP.AuerandL.Wei,279–313.Berlin:MoutonDeGruyter.Giesbers,H.1989.Code-switchingtussenDialectenStandaardtaal.Amsterdam:P.J.Meertens-Instituut.Gong,W.2005.Englishincomputer-mediatedenvironments:AneglecteddimensioninlargeEnglishcorpuscompilation.ProceedingsofCorpuslinguisticsconferenceseries1(1).[http://www.birmingham.ac.uk/research/activity/corpus/publications/conference-archives/2005-conf-e-journal.aspx](25.09.2016)Gumperz,J.J.1982.Languageandsocialidentity.Vol.2.Cambridge:CambridgeUniversityPress.Jefferson,G.,SacksH.&Schegloff,E.A.1987.“Notesonlaughterinthepursuitofintimacy”.InG.Button&J.R.E.Lee(eds.),TalkandSocialOrganization.Clevedon:MultilingualMatters,152–205.Isaacs,E.A.&Tang,J.C.1994.Whatvideocanandcannotdoforcollaboration:acasestudy.MultimediaSystems 2.63-73.Jenkins,J.(2015).RepositioningEnglishandmultilingualisminEnglishasaLinguaFranca.EnglishesinPractice2(3),49-85.Klimpfinger,T.(2009).“She’smixingthetwolanguagestogether”- Formsandfunctionsofcode-switchinginEnglishasalingua franca.InA.Mauranen&E.Ranta(eds.),Englishasalinguafranca:studiesandfindings.Newcastle:CambridgeScholarsPress,348–372.

References

Page 35: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Laitinen,M..2015.Fromdiachronytotypology:testingtypologicalprofilingonadvancednon-nativedata.ChangingEnglish2015:Integratingcognitive,social&typologicalperspectives,8-10June,2015,Helsinki.Levinson,S.C.1983.Pragmatics.Cambridge:CambridgeUniversityPress.Mauranen,A.(2012).ExploringELF:AcademicEnglishShapedbyNon-NativeSpeakers.Cambridge:CambridgeUniversityPress.MICASE.2002.TheMichiganCorpusofAcademicSpokenEnglish.R.C.Simpson,S.L.Briggs,J.Ovens,andJ.M.Swales;projectleader: UteRömer.AnnArbor,MI:TheRegentsoftheUniversityofMichigan.Nelson,G.2008.ICEmarkupmanualforspokentexts.[http://ice-corpora.net/ice/spoken.doc](25.09.2016).Norrick,N.etal.2013.Transcriptionconventions.Saarbrücken:SaarlandUniversity.[http://www.uni-saarland.de/fileadmin/user_upload/Professoren/fr43_Engling/downloads/trans_conv.pdf](30.06.2016).Ogden,C.K.&Richards,I.A.(1936).Themeaningofmeaning.Astudyoftheinfluenceoflanguageuponthoughtandofthescienceofsymbolism.London:Routledge&Kegan.Olson,G.M.&Olson,J.S..2000.Distancematters.Human-ComputerInteraction 15(2-3).139-178.Pennycook,A.2010.Languageasalocalpractice.Oxford:Routledge.Pitzl,M.-L..2009.Weshouldnotwakeupanydogs.IdiomandmetaphorinELF.InA.Mauranen,A&E.Ranta(eds.),Englishasalinguafranca:studiesandfindings,298–322.Newcastle:CambridgeScholarsPress.Poplack,S.(1980).’SometimesI’llstartasentenceinSpanishyterminoenespañol’:towardatypologyofcode-switching.Linguistics18(7/8),581–618.Poplack,S.(1988).Contrastingpatternsofcode-switchingintwocommunities.InM.Heller(ed.).Codeswitching:Anthropologicalandsociolinguisticperspectives(Vol.48).Berlin:deGruyter.215-244.Poulisse,N.&Bongaerts,T.(1994).Firstlanguageuseinsecondlanguageproduction.Appliedlinguistics,15(1),36-57.Romaine,S.(2001).Bilingualism.Oxford:Blackwell.

References

Page 36: Paralanguage and gesture in a corpus of Skype ...Paralanguage and gesture in a corpus of Skype conversations: “ ... Knödel ((German (0.8))),.. ... and I like have to give one lesson

Sebba,M.2009.Onthenotionsofcongruenceandconvergenceincode-switching.InB.E.Bullock&A.J.Toribio(eds.),TheCambridgehandbookoflinguisticcode-switching.Cambridge:CambridgeUniversityPress,40–57.Sauer,S.&Lüdeling,A..2016.Flexiblemulti-layerspokendialoguecorpora.InternationalJournalofCorpusLinguistics21(3):421-440.Amsterdam:Benjamins.Seidlhofer,B.2003.AconceptofinternationalEnglishandrelatedissues:From‘realEnglish’to‘realisticEnglish.’Strasbourg:CouncilofEurope,LanguagePolicyDivision.Schmidt,S.;Brunner,M.-L.&Diemer,S..2014.CASE:CorpusofAcademicSpokenEnglish:TranscriptionConventions. Saarbrücken:SaarlandUniversity.[http://www.uni-saarland.de/index.php?id=48506](25.09.2016).Siegel,J.1995.HowtogetalaughinFijian:Code-switchingandhumor.LanguageinSociety,24(01),95-110.Spencer-Oatey,H.(2000).RapportManagement:AFrameworkforAnalysis.InH.Spencer-Oatey(Ed.),CulturallySpeaking:ManagingRapportThroughTalkAcrossCultures.London:Continuum,11-46.Spencer-Oatey,Helen.2002.“Managingrapportintalk:Usingrapportsensitiveincidentstoexplorethemotivationalconcernsunderlyingthemanagementofrelations”.JournalofPragmatics 34(5):529-545.Tang,J.C.&Isaacs,E.1993.Whydouserslikevideo?Studiesofmultimedia-supportedcollaboration.ComputerSupportedCooperativeWork:TheJournalofCollaborativeComputing1.163-196.Temmerman,M.(2009).Communicativeaspectsofdefinitionsinclassroominteraction:Learningtodefineinclassforfirstandsecondlanguagelearners.LinguisticsandEducation 20,126-144.Thompson,Paul.2005.Spokenlanguagecorpora.InM.Wynne(ed.),DevelopingLinguisticCorpora:AGuidetoGoodPractice[http://users.ox.ac.uk/~martinw/dlc/chapter5.htm](25.09.2016).Vettorel,P.(2014).ELFinwidernetworking:Bloggingpractices.Berlin:DeGruyterMouton.VOICE. 2013.TheVienna-OxfordInternationalCorpusofEnglish(version2.0XML).2013.Director:B.Seidlhofer.UniversityofVienna.[https://www.univie.ac.at/voice/] (25.09.2016).Wei,L.(1998).Thewhyandhowquestionsintheanalysisofconversationalcode-switching.InP.Auer(ed.),Code-switchinginconversation:Language,interactionandidentity.LondonandNewYork:Routledge,156–179.

References