What is the Imitation Game? Robert Evans and Harry Collins Centre for the Study of Knowledge,...
-
Upload
richard-harrell -
Category
Documents
-
view
220 -
download
0
Transcript of What is the Imitation Game? Robert Evans and Harry Collins Centre for the Study of Knowledge,...
What is the Imitation Game?
Robert Evans and Harry Collins
Centre for the Study of Knowledge, Expertise and Science (KES)Cardiff School of Social Sciences
http://www.cardiff.ac.uk/socsi/expertise
Overview
Theory and Background Different Kinds of Expertise Turing Test
Imitation Games in Practice Software and data Identify and Chance conditions Qualitative and Quantitative Results
Comparative Research Cross-cultural and longitudinal research
Three Kinds of Expertise
Contributory expertise: enables those who have acquired it to contribute linguistically and practically to the community through the expertise is sustained. The most common usage of the word ‘expert’.
Interactional Expertise: expertise in the language of a specialism in the absence of expertise in its practice. Like contributory expertise, it requires the tacit-knowledge acquired by immersion in a form-of-life (i.e. socialisation). It enables individuals to talk as if they had contributory expertise even though they lack practical or craft skills.
No Expertise: not even the ability to talk intelligently about a given domain or topic
Collins and Evans (2002)
Turing Tests and Imitation Games
In the Turing Test, the judge must decide which is the
computer and which is the human.
In the Imitation Game, the judge must decide which participant shares their social group.
Modern Imitation Game
Female judge setting questions
Female answering naturally
Male pretending to be female
How often do you pluck your eyebrows?
R2‘not very often, when
they need doing’
R1‘once a week’
R2 is female ‘because I expected the man to believe women are more regulated in their beauty regime than they actually are
Judge compares answers
Judge’s question
Respondents Answers
Judge’s verdict
Judge’s confidence
Judge’s reason
Automatically recorded data
Software records Questions and answers
analysis of topics and assumptions used by judges (i.e. what is distinctive about own culture)
Guesses and confidences what discriminates and what does not (i.e.
what is shared with others and what is not) Judge’s reasons
cultural norms and variation (i.e. what counts as a ‘mistake’ and what as normal variation)
Researcher generated data
Researchers needed to record Problems with method and software
(e.g. how to use, do participants understand role etc)
Strategies used by judges and pretenders (e.g. what is hypothesis being tested by judge, how does pretender ‘know’ the answer)
Effort and determination (e.g. judge’s motives and fears)
Imitation Games with the Blind
Identify Condition Judge is blind Target expertise is
‘being blind’ Sighted participant
has to pretend to be blind
Prediction: Judge will succeed in identifying participants
Chance Condition Judge is sighted Target expertise is
‘being sighted’ Blind participant has
to pretend to be sighted
Prediction: Blind person will succeed in pretending
Sighted dialogs
Judge R1 R2
Could you tell me roughly how old you are and whether you have been registered blind since birth
I'm 50 and have been blind since I was 10
I'm 30 and I've been registered blind since I was
Do you have any residual sight and what mobility aids do you use?
No I have no residual sight. I use a white stick and have a guide dog
Twelve. I've got light and dark and color perception in one eye and I use a guide dog.
Judges’ reasons
Question One The second person is not black and white and you
do not usually lose your sight over-night, so the fact they mention being registered suggest that they are blind. If the first one was blind they would normally say how they became blind if it was sudden. (level 2)
Question Two I have both white stick and dog but would never use
both at same time. Therefore if I was responding I would say something like I use a guide dog predominantly but sometimes use a white stick -- but if you are blind you would call it a cane normally. Also, number 2 was much less black and white. It's always grades of blindness. (level 4)
Blind dialogs
Judge R1 R2
So let me start with sport. Are you interested in tennis and do you ever watch it on the television?
I watch Wimbledon a little bit on the television and occasionally the Australian Open in January
I like tennis but only watch big tournaments like Wimbledon
So tell me what you think about the Hawk-Eye line judging system
Not being a tennis professional it is not for me to say if it should or should not be used. It does not really alter viewing
It adds an other element to the game which could make it more interesting
Blind dialogs
Judge R1 R2
How accurately would you say a human can judge the flight of a tennis-ball? I mean, would you say they could tell the difference between touch the line and 1mm out 2mm out 1 cm out, 2 cm out, or what, and what would it depend on?
I think often a tennis player is not in a position to judge accurately as they are not usually parallel with the line. I think that if you set up a test for a line judge with two balls one which landed on the line and one which landed 1mm away from the line, I don't think they could tell the difference. If you think how small 1mm is then it would be so hard for them to judge.
it would depend on the speed the ball was travelling and the position of the judge relative to the line and obviously the closer the ball is the line the harder it would be to make a judgement. So you would have to judge each call on an individual bases as there are a lot of factors.
Judges’ reasons
R1 is the blind person ‘I think respondent 1 gives himself away when he
discusses the human judgments on the flight of a tennis ball.’
‘I cannot believe a sighted person saying that Hawk-eye does not alter the viewing.’
R2 is the blind person ‘The Hawk-Eye questions reveal some quite specific
information that I don’t think was published in audio media. Also, the story wasn’t that important that I’d expect it to be picked up by the audio news services provided to the blind.’
‘person 2 seems really unfamiliar with hawk-eye, given that they say they watch Wimbledon’
Identify Conditions
0.33
0.730.86
0.44
0.68
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
colour blind pitch blind sexuality religion
Topics
Ide
nti
fic
ati
on
Ra
tio
ID Ratio Net Don't Know
Comparisons across topics
Mean IR for ID condition approx 0.6
Mean IR for Chance condition approx zero
Chance Conditions
0.05 0.000.13
-0.01-0.13
-0.20
0.00
0.20
0.40
0.60
0.80
1.00
colour blind pitch blind sexuality religion
Topics
Ide
nti
fic
ati
on
Ra
tio
ID Ratio Net Don't Know
IG and Comparative Research
Social Change and Social Difference How to standardise data collection? How to reflect local variations?
Imitation Game as ‘ethno-method’ Participants’ discourse is always contemporary Judge as proxy ethnographer Participants as proxy stranger Participant errors like breaching experiments
Imitation Game as secondary data Identification ratios by time, place and topic Qualitative data based on topics, reasons, errors
Imitation Games on Gender
Hypotheses Malestream culture: hegemonic
masculinity implies asymmetrical results
1960s and all that: gender roles and identities more flexible so knowledge widely shared
Age as confounding factor Younger people have less interaction The 1960s are history?
Research Design
Research Methods module project Recruit students and heterosexual,
cohabiting parents Phase One
Students: 29 games (15 F, 14 M) Parents: 25 games (12 F, 13 M)
Phase Two Students: 70 games (40 F, 30 M) Parents: 70 games (50 F, 20 M)
Results by gender Female Male
Incorrect 11 12
Don't Know 15 10
Correct 29 22
Total 55 44
IR 0.33 0.23
Female Male
Incorrect 21 13
Don't Know 19 9
Correct 22 11
Total 62 33
IR 0.02 -0.06
Phase One and Two, students
Approximates identify condition for both groups, female judges appear more successful than male judges but not statistically significant (p = 0.37)
Phase One and Two, students
Approximates chance condition for both groups, no statistically significant difference between genders (p = 0.26)
Results by age Young Old
Incorrect 6 9
Don't Know 5 4
Correct 18 12
Total 29 25
IR 0.41 0.12
Young Old
Incorrect 17 25
Don't Know 20 24
Correct 33 21
Total 70 70
IR 0.23 -0.06
Phase One only
Clear difference between groups; young judges more successful than older judges (p = 0.031)
Phase Two only
Clear difference between groups; young judges more successful than older judges (p = 0.007)
Conclusions for gender
Differences by gender No statistically significant differences in sample
as a whole Some differences in topics chosen to
‘represent’ and test cultures Clear differences by age
Older participants much better at pretending (i.e. have more interactional expertise)I
Impossible to distinguish between ‘1960s’ hypothesis and the ‘learning’ hypothesis without longitudinal study
Conclusions
Interactional expertise Useful concept for explaining social phenomena
Imitation Game Combines qualitative and quantitative
approaches Opportunities for comparative research within
cultures, across cultures, over time Want to know more
http://www.cardiff.ac.uk/socsi/expertise
References and reading
Collins, Harry and Robert Evans (2002) ‘The Third Wave of Science Studies: Studies of Expertise and Experience’, Social Studies of Sciences, 32 (2): 235-96.
Collins, Harry and Robert Evans (2007) Rethinking Expertise, Chicago, IL: The University of Chicago Press.
Collins, Harry, Robert Evans, Rodrigo Ribeiro and Martin Hall (2006), ‘Experiments with Interactional Expertise, Studies In History and Philosophy of Science, Volume 37, No. 4 (Dec 2006), pp. 656-674.
Evans, Robert and Harry Collins (forthcoming, 2010) ‘Interactional Expertise and the Imitation Game’ in Michael Gorman (ed) Trading Zones and Interactional Expertise: Creating New Kinds of Collaboration, Chicago, IL: MIT Press