Mike Scott, School of EnglishUniversity of Liverpool

Keyness 1

…challenges in

investigating Keyness

NLP Group Computer Science

University of Sheffield

15 February 2008

Purpose

1. To explore the notion of keyness2. and its implications in corpus-based

study3. and to consider concgrams4. and key concgrams5. all with reference to WordSmith

Keyness 2

Overview

Keyness, as a new territory, looks promising and has attracted colonists and prospectors. It generally appears to give robust indications of the text’s aboutness together with indicators of style.

Keyness 3

the text’s aboutness

Keyness 4

Issuesthe issue of text section v. text v. corpus

v. sub-corpusstatistical questions: what exactly can

be claimed?how to choose a reference corpushandling related forms such as

antonyms

Keyness 5

Machine and Human KWSRigotti and Rocci (2002) warn that

machine identification of key words omits all interpretation of the writer’s intentions, cannot get at cultural implications and does not spot the congruity of the meanings of each section with the next.

Keyness 6

metaphors“In our view, a natural language text, slippery

and vague as it may be, is not a stone soup where words float free, tied only to their multiple associations within a Foucoultian discourse” (Rigotti and Rocci, 2002)

Keyness 7

Of course it doesn’t actually understand…

Keyness 8

… or know what is “correct”

Keyness 9

… only look at what is found in text

Keyness 10

… or context

… whether marked up or not …

Context?

Keyness 11

Keyness 12

Levels of Context

P hysica l env ironm ent

If so

what is the status of the “key words” one may identify and what is to be done with them?

Keyness 13

Issues1. the issue of text section v. text v. corpus

v. sub-corpus

2. statistical questions: what exactly can be claimed?

3. how to choose a reference corpus

4. handling related forms such as antonyms

5. what is the status of the “key words” one may identify and what is to be done with them?

Keyness 14

text section v. text v. corpus v. sub-corpus

text section: levels 1-5text: level 6corpus: levels 7 & 8

Keyness 15

But these are often not clearly differentiated

“text”, level 6: with or without mark-up, images, sounds?

what do we mean by section, chapter (4) and other non linguistically defined categories?

is text itself mutating?

Keyness 16

Internet text

Keyness 17

Wikipedia homepage (part)

Keyness 18

Wikipedia homepage (part)

Keyness 19

Wikipedia article(3 parts of same article)

Keyness 20

Wikipedia discussionfrom History of the stall articlelatest contributor, “Talk” section

Keyness 21

statistical issuesp value is a well-established standard, relying

on the notion of chance, random effectsbut

if you run lots of comparisons some will spuriously (by chance) appear significant

if we’re operating at the level of word or cluster, text itself doesn’t consist of randomly ordered words

Keyness 22

Implicationthere is no statistical defence of the whole set

of KWsbut only of each onecomparing KW p values is not advisable

Keyness 23

the South Utsere farmer3 crops

bananasbarleychick-peas

Keyness 24

3 problemsstormwind

drought

choosing a reference corpususing a mixed bag RC, the larger the RC the better but

a moderate sized RC may suffice. the keyword procedure is fairly robust. KWs identified even by an obviously absurd RC can be

plausible indicators of aboutness, which reinforces the conclusion that keyword analysis is robust.

genre-specific RCs identify rather different KWsthe aboutness of a text may not be one thing but

numerous different ones.

Scott (forthcoming)

Keyness 25

related forms such as antonyms

Keyness 26

status of the “key words”

Keyness 27

Concgram section

Keyness 28

What is a “concgram”?For years it has been easy to search for or

identify consecutive clusters (n-grams) such asAT THE END OFMERRY CHRISTMASTERM TIME.

It has also been possible to find non-consecutive linkages such as STRONG within the horizons of TEA by adapting searches to find context words.

In such cases we might get...STRONG blah blah blah TEA......TEA blah blah blah blah STRONG...etc.

29

The concgram procedure takes a whole corpus of text and finds all sorts of combinations like

...STRONG blah blah blah TEA... ...TEA blah blah blah blah STRONG... ...STRONG TEA... ...TEA STRONG...

whether consecutive or not...

a sequence (n-gram)within a concordance span. (“skip-gram” is used (Wilks 2005) to describe non-

contiguous word associations but doesn’t include TEA […] STRONG)

30

Cheng, Greaves & Warren (2006)For our purposes, a ‘concgram’ is all of the

permutations of constituency variation and positional variation generated by the association of two or more words. This means that the associated words comprising a particular concgram may be the source of a number of ‘collocational patterns’ (Sinclair 2004:xxvii). In fact, the hunt for what we term ‘concgrams’ has a fairly long history dating back to the 1980s (Sinclair 2005, personal communication) when the Cobuild team at the University of Birmingham led by Professor John Sinclair attempted, with limited success, to devise the means to automatically search for non-contiguous sequences of associated words.Cheng, Greaves & Warren (2006:414)

31

ConcGram (©) aims to be"a search-engine, which on top of the

capability to handle constituency variation (i.e. AB, ACB), also handles positional variation (i.e. AB, BA), conducts fully automated searches, and searches for word associations of any size." (2006:413)

WSConcGram is developed in homage to this idea.

32

The goal

33

Cheng, Greaves & Warren (2006:426)

A ProblemGreaves (2007) reported that ConcGram

requires months using numerous linked PCs to generate the 5-word concgrams based on a corpus of some 5 million words.

There are a lot of combinations to take into account…

34

Implementation in WordSmithThe plan: to process a corpus of adequate size (say

10 or more million words)find all instances of all frequently co-occurring

wordsco-occurring within a given spanidentify them as potential

pairs (sup ... with)triplets (light ... an ... dark)quadruplets (with ... the ... of ... war)quintuplets (eyes ... are ... full ... of ... tears) etc. (a ... light ... condition ... in ... beauty ... dark)

determine whether they are significantly associated35

Stagesdesign the procedureplan human interface aspects

36

Procedures and Routines

WS3-5’s WordList index function already knew how to process a corpus and identify all instances of each word in it...

37

Index“A kingdom for a stage, princes to act

And monarchs to behold the swelling scene.” (Henry V)

38

Huge file of records containing token data:word_type_number,next_known_token,file_number,file_byte_position

R1, R2, R3 ... RNR1’s word_type_number=1; (a)R2’s word_type_number=2; (kingdom)R3’s word_type_number=3; (for)R4’s word_type_number=1; (a)R5’s word_type_number=4; (stage)R6’s word_type_number=5; (princes)R7’s word_type_number=6; (to)R8’s word_type_number=7; (act)R9’s word_type_number=8; (and)R10’s word_type_number=9; (monarchs)R11’s word_type_number=6; (to)

Huge file of records containing token data:word_type_number,next_known_token,file_number,file_byte_position

R1, R2, R3 ... RNR1’s word_type_number=1; (a)R2’s word_type_number=2; (kingdom)R3’s word_type_number=3; (for)R4’s word_type_number=1; (a)R5’s word_type_number=4; (stage)R6’s word_type_number=5; (princes)R7’s word_type_number=6; (to)R8’s word_type_number=7; (act)R9’s word_type_number=8; (and)R10’s word_type_number=9; (monarchs)R11’s word_type_number=6; (to)

WSConcGram procedures (1)process the index, looking at each instance of

words above a certain threshold frequency (e.g. 5 instances)

considering all its neighbours within a given span (e.g. 5)

finding all pairs repeated more than a threshold number of times

saving in a file data on where in the corpus each pair is to be found

39

WSConcGram procedures (2)

sort the file of pairsprocess it, finding overlaps,

e.g. where HOW and MATTER and NOW are all found within the default span

sorting the resulting triplets, quadruplets, etc and

storing them in another file

40

WSConcGram Filesshakespeare.typesshakespeare.tokensshakespeare.base_pairsshakespeare.base_indexshakespeare.base_index_cg

41

Human Interface Aspectssorting concgrams by frequency and alphabeticallydisplaying the root word typeschoosing concgram formsclustering them in treesfiltering according to

statistical propertiesrequired word(s)

other needscopy, save as .txt, print, print previewconcordance selected concgrams

42

Problem areascomputing the association statistics correctlyclustering each concgramshowing & hiding parts of a concgramordering them in a tree structure

43

References Berber Sardinha, Tony, 1999. Using Key Words in Text Analysis: practical

aspects. DIRECT Papers 42, LAEL, Catholic University of São Paulo. Berber Sardinha, Tony, 2004. Lingüística de Corpus. Barueri: Manole. Cheng, Winnie, Chris Greaves & Martin Warren, 2006. “From n-gram to

skipgram to concgram”, International Journal of Corpus Linguistics, Vol. 11, No. 4, pp. 411-433.

Greaves, Chris, 2007. Demonstration of ConcGram. Keyness in Text conference, Certosa di Pontignano, Tuscany, Italy, 26-30 June 2007.

Culpeper, J. ,2002. 'Computers, language and characterisation: An Analysis of six characters in Romeo and Juliet'. In: U. Melander-Marttala, C. Östman and M. Kytö (eds.), Conversation in Life and in Literature: Papers from the ASLA Symposium, Association Suedoise de Linguistique Appliquée (ASLA), 15. Universitetstryckeriet: Uppsala, pp.11-30.

Kemppanen, Hannu 2004. Keywords and Ideology in Translated History Texts: A Corpus-based Analysis. Across Languages and Cultures 5 (1), 89-106

Rigotti, Eddo and Andrea Rocci, 2002. From Argument Analysis to Cultural Keywords (and back again). http://www.ils.com.unisi.ch/articoli-rigotti-rocci-keywords-published.pdf (accessed May 2007). In F. H. van Eemeren et al, Proceedings of the 5th Conference of the International Society for the Study of Argumentation. Amsterdam: SicSat. pp. 903-908.

Scott, M., 1996 with new versions in 1997, 1999, 2004, Wordsmith Tools, Oxford: Oxford University Press.

Scott, M., 1997a. "PC Analysis of Key Words -- and Key Key Words", System, Vol. 25, No. 1, pp. 1-13.

Scott, M., 1997b. "The Right Word in the Right Place: Key Word Associates in Two Languages", AAA - Arbeiten aus Anglistik und Amerikanistik, Vol. 22, No. 2, pp. 239-252. 44

References Scott, M., 2000a. ‘Focusing on the Text and Its Key Words’, in L. Burnard & T. McEnery

(eds.), Rethinking Language Pedagogy from a Corpus Perspective, Volume 2. Frankfurt: Peter Lang., pp. 103-122.

Scott, M. 2000b. Reverberations of an Echo, in B. Lewandowska-Tomaszczyk & P.J. Melia (eds.) PALC’99: Practical Applications in Language Corpora. Lodz Studies in Language, Volume 1. Frankfurt: Peter Lang., pp. 49-68.

Scott, M., 2001. ‘Mapping Key Words to Problem and Solution’ in M. Scott & G. Thompson (eds.) Patterns of Text: in honour of Michael Hoey, Amsterdam: Benjamins, pp. 109-127.

Scott, M., 2002. ‘Picturing the key words of a very large corpus and their lexical upshots – or getting at the Guardian’s view of the world’ in B. Kettemann & G. Marko (eds.) Teaching and Learning by Doing Corpus Analysis, Amsterdam: Rodopi, pp. 43-50 and cd-rom within the cover of the book.

Scott, M. 2006. "The Importance of Key Words for LSP" in Arnó Macià, E., A. Soler Cervera & C. Rueda Ramos (eds.), Information Technology in Languages for Specific Purposes: issues and prospects. New York: Springer, pp. 231-243.

Scott. M. (forthcoming) In Search of a Bad Reference Corpus. AHRC Methods Network. Scott, M. & Tribble, C., 2006. Textual Patterns: keyword and corpus analysis in

language education, Amsterdam: Benjamins. Seale C, Charteris-Black J, Ziebland S. 2006. Gender, cancer experience and internet

use: a comparative keyword analysis of interviews and online cancer support groups. Social Science and Medicine. 62, 10: 2577-2590

Tribble, Chris, 1999, "Genres, keywords, teaching: towards a pedagogic account of the language of project proposals" in L. Burnard & A. McEnery (eds.) Rethinking Language Pedagogy from a Corpus Perspective: Papers from the Third International Conference on Teaching and Language Corpora, (Lodz Studies in Language). Hamburg: Peter Lang.

Wilks, Yorick, 2005. REVEAL: the notion of anomalous texts in a very large corpus. Tuscan Word Centre International Workshop. Certosa di Pontignano, Tuscany, Italy, 31 June–3 July 2005 (cited in Cheng et al.)

Keyness 45