The Age of Intelligent Machines _ a (Kind of) Turing Test _ Kurzweilai-20
13
KurzweilAI | Accelerating Intelligence. e-book: The Age of Intelligent Machines THE AGE OF INTELLIGENT MACHINES | A (Kind Of) Turing Test September 24, 2001 author | Ray Kurzweil No ideas but in things. William Carlos Williams As discussed in several of the contributed articles in this book, the Turing test was devised by Alan Turing as a way of certifying machine intelligence. Turing described a situation in which a human judge communicates with both a computer and a human using a computer terminal. The judge’s task is to determine which is which. The judge cannot see the computer or the human and must make his or her determination by interviewing both. The computer attempts to trick the judge into selecting it as the human. The essence of the Turing Test is that the computer attempts to act like a human within the context of an interview over terminal lines. A narrower concept of a Turing test is for a computer to successfully imitate a human within a particular domain of human intelligence. We might call these domain-specific Turing tests. One such domain-specific Turing test, based on a computer’s ability to write poetry, is presented here. The Kurzweil Cybernetic Poet is a computer program (written by the author) and provided with an input file of poems written by a human author or authors. The program analyzes these poems and creates a word-sequence model based on the poems it has just read. It then writes original stanzas of poetry using the model it has created. Some of the following stanzas of poetry were written by the Kurzweil Cybernetic Poet. Some were written by human authors (in fact the same human authors that were read and analyzed by the Kurzweil Cybernetic Poet). See if you can tell which are which. On a piece of paper, write down the numbers 1 through 28. Then put a C by that number if you believe that the corresponding st anza was written by the computer. Put an H if you believe the stanza was written by a human poet. The answers are in a footnote. 1 Following the answers are the results obtained when the test was taken by 16 human (both adult and child) judges. 1. is beauty itself that they were walking there. All along the new world naked,
description
The Age of Intelligent Machines _ a (Kind of) Turing Test _ Kurzweilai-20
Transcript of The Age of Intelligent Machines _ a (Kind of) Turing Test _ Kurzweilai-20
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 1/13
KurzweilAI|AcceleratingIntelligence.ebook:TheAgeofIntelligentMachines
THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest
September24,2001
author|RayKurzweil
Noideasbutinthings.
WilliamCarlosWilliams
Asdiscussedinseveralofthecontributedarticlesinthisbook,theTuringtestwasdevisedbyAlanTuringasawayofcertifyingmachineintelligence.Turingdescribedasituationinwhichahumanjudgecommunicateswithbothacomputerandahumanusingacomputerterminal.Thejudgestaskistodeterminewhichiswhich.Thejudgecannotseethecomputerorthehumanandmustmakehisorherdeterminationbyinterviewingboth.Thecomputerattemptstotrickthejudgeintoselectingitasthehuman.
TheessenceoftheTuringTestisthatthecomputerattemptstoactlikeahumanwithinthecontextofaninterviewoverterminallines.AnarrowerconceptofaTuringtestisforacomputertosuccessfullyimitateahumanwithinaparticulardomainofhumanintelligence.WemightcallthesedomainspecificTuringtests.OnesuchdomainspecificTuringtest,basedonacomputersabilitytowritepoetry,ispresentedhere.
TheKurzweilCyberneticPoetisacomputerprogram(writtenbytheauthor)andprovidedwithaninputfileofpoemswrittenbyahumanauthororauthors.Theprogramanalyzesthesepoemsandcreatesawordsequencemodelbasedonthepoemsithasjustread.Itthenwritesoriginalstanzasofpoetryusingthemodelithascreated.SomeofthefollowingstanzasofpoetrywerewrittenbytheKurzweilCyberneticPoet.Somewerewrittenbyhumanauthors(infactthesamehumanauthorsthatwerereadandanalyzedbytheKurzweilCyberneticPoet).Seeifyoucantellwhicharewhich.Onapieceofpaper,writedownthenumbers1through28.ThenputaCbythatnumberifyoubelievethatthecorrespondingstanzawaswrittenbythecomputer.PutanHifyoubelievethestanzawaswrittenbyahumanpoet.Theanswersareinafootnote.1Followingtheanswersaretheresultsobtainedwhenthetestwastakenby16human(bothadultandchild)judges.
1.isbeautyitself
thattheywerewalkingthere.Allalongthenewworldnaked,
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 2/13
cold,familiarwind
2.Pinkconfusedwithwhite
flowersandflowersreversed
takeandspilltheshadedflame
dartingitback
intothelampshorn
3.Thewindsoftheoozywoodswhichwear
theocean,withazuremossandflowers
Sosweet,thepurpleeven
Isleepinthearrows
Ofthedomeofdeath.
4.0thou,
WhomovedamongsomefierceMaenad,evenamongnoise
andblue
Betweenthebonessang,scatteredandthesilentseas.
5.Sheeyesmewithaningrowneye,
intherhythmofteacuptapping
thinksofsweepingawaycrumbs
6.AtsixIcannotpray:
Prayforlovers,
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 3/13
throughnarrowstreets
Andpraytofly
ButtheVirginintheirdarkwintrybed
7.Whatseaswhatshoreswhatgraniteislandstowardmytimbers
Andwoodthrushcallingthroughthefog
Mydaughter.
8.Imaginenowatreeinwhitesailsstillwhirled
Abouttheleaves
willbeofsilences
Calmandangels
9.andthesun,dippingintotheavenues
streakingthetopsof
theirregularredhouselets,
and
thegayshadowsdroppinganddropping.
10.Themorningandalready
aperfectifslightlypaled
oldparkturnedwithyoungwomen
seizedinamber.
11.Interestingbook?
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 4/13
shesits
dancingbytheelectrictypewriter,
bloodlessrevolutionofmeats
stringsofuse,
Politic,cautious,andthefact
sheisthefact
sheiscallingthemall
Thechildrenathisfeet
heisalwaystime
Torollitwasdark,
damp,jagged,likethevoice
Becauseofloveends.
12.Menwithpickedvoiceschantthenames
ofcitiesinahugegallery:promises
thatpullthroughdescendingstairways
toadeeprumbling.
13.Wherewerethou,sadHour,selectedfromwhoseraceis
guidingme,
LuredbytheloveofAutumnsbeing,
Thou,fromheavenisgone,wherewaslornUrania
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 5/13
Whenrockedtoflywiththeeinherclarionoerthearmsofdeath.
14.LadyofAutumnsbeing,
Thou,fromtheday,havingtocare
Teachusnowthoroughlysmallandcreate,
Andthenpresume?
Andthis,andme,
Andplaceoftheunspokenword,theunreadvisioninBaiaesbay,
AndtheposterityofMichelangelo.
15.Iamlonely,lonely.
Islapananswermyself
shehidesdeepwithinher
yetplays
Milkless
16.0myshoulders,flanks,buttocks
againsttrespassers,
againstthieves,
storms,sun,fire,
againstthieves,
storms,sun,fire,
againstflies,againstweeds,stormtides,
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 6/13
neighbors,weaselsthatwaken
Thesilentseas.
17.thedays,lockedineachothersarms,
seemstill
sothatsquirrelsandcoloredbirds
goaboutateaseover
thebranchesandthroughtheair.
18.Iamwatchingantsdigtunnelsandburythemselves
theygowithoutwaterorlove
19.Ladyissick,
perhapsvomiting,
perhapslaboring
totheusualreign
20.Rainissweet,brownhair
Distraction,musicinpassageways.
Sixoclock.
Thetime.Redeem
Theworldandwaking,wearing
21.Wipeyourhandacrossyourmouth,andlaugh
Theworldsrevolvelikeancientwomen
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 7/13
Gatheringfuelinvacantlots.
22.Ishouldhavebeenapairofraggedclaws
Scuttlingacrossthefloorsofsilentseas.
23.patchesofall
savebeauty
therigidwheeltracks.
Theroundsun
thebed.
Shesmiles,Yes
youpleasefirst
thenstays
withherselfalone
andthendividingandover
andsplashedandafteryouare
listeninginhereyes
24.Allalongtheroadthereddish
purplish,forked,upstanding,twiggy
stuffofbushesandsmalltrees
withdead,brownleavesunderthem
leaflessvines
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 8/13
25.Prayforthosewhoarebranchesonforever
26.Likeasodofwar
housesofsmall
whitecurtains
Smellofshimmering
ashwhite,
anaxe
27.Byactionorbysuffering,andwhosehour
Wasdrainedtoitslastsandinwealorwoe,
Sothatthetrunksurvivedbothfruitorflower
28.isasteadyburning
theroadthebattlesfury
cloudsandashandwaning
sendingout
youngpeople,
Theabove28questionpoeticTuringtestwasadministeredto16humanjudgeswithvaryingdegreesofcomputerandpoetryexperienceandknowledge.The13adultjudgesscoredanaverageof59percentcorrectinidentifyingthecomputerpoemstanzas,68percentcorrectinidentifyingthehumanpoemstanzas,and63percentcorrectoverall.Thethreechildjudgesscoredanaverageof52percentcorrectinidentifyingthecomputerpoemstanzas,42percentcorrectinidentifyingthehumanpoemstanzas,and48percentcorrectoverall.
Thechartsshowtheactualscoresobtainedbythe16humanjudgesasbrokendownbyadult/child,computerexperience,andpoetryexperience.Ascanbeseenfromthecharts,therewerenotrendsbasedonlevelofcomputerexperienceorpoetryexperienceclearlydiscerniblefromthislimitedsample.Theadultsdidscoresomewhatbetterthanthechildren.Thechildrenscoredessentiallyatchancelevel(approximately50percent)andtheadultsachievingslightlybetterthanchance.
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 9/13
Thenextchartshowsthenumberofcorrectandincorrectanswersforeachofthe28poemsorstanzas.Whiletheadultjudgesscoredsomewhatbetterthanchance(63percent),theiranswerswerefarfromperfect.Thecomputerpoetwasabletotrickthehumanjudgesmuchofthetime.Someofthecomputerpoems(numbers15and28,forexample)wereparticularlysuccessfulintrickingthejudges.
WecanconcludethatthisdomainspecificTuringtesthasachievedsomelevelofsuccessintrickinghumanjudgesinitspoetrywritingability.Amoredifficultproblemthanwritingstanzasofpoetryiswritingcompletepoemsthatmakethematic,syntactic,andpoeticsenseacrossmultiplestanzas.AfutureversionoftheKurzweilCyberneticPoetiscontemplatedthatattemptsthismoredifficulttask.Tobesuccessful,themodelscreatedbytheCyberneticPoetwillrequirearicherunderstandingofthesyntacticandpoeticfunctionofeachword.
EventheoriginallyproposedTuringtestinvolvingterminalinterviewsisnotablyimpreciseindeterminingwhenthecomputerhasbeensuccessfulinimitatingahuman.Howmanyjudgesneedtobefooled?Atwhatscoredoweconsiderthehumanjudgestohavebeenfooled?Howsophisticateddothejudgesneedtobe?Howsophisticated(orunsophisticated)doesthehumanfoilneedtobe?Howmuchtimedothejudgeshavetomaketheirdetermination?ThesearebutafewofthemanyquestionssurroundingtheTuringtest.(ThearticleACoffeehouseConversationontheTuringTestbyDouglasHofstadterinchapter2providesanentertainingdiscussionofsomeoftheseissues).ItisclearthattheeraofcomputerspassingtheTuringtestwillnothappensuddenly.OncecomputersstarttoarguablypasstheTuringtest,thevalidityofthetestsandthetestingprocedureswillundoubtedlybedebated.ThesamecanbesaidforthenarrowerdomainspecificTuringtests.
Adultscoresonpoemstanzascomposedbyacomputer(13adults,%correct) LevelofComputerexperienceLevelofpoetryexperience
Little Moderate Professional AverageLittle 56 44,69,75 63,75 64Moderate 50,56,63 56,63 75 61Alot 25 25Average 56 61 59 59Childrensscoresonpoemstanzascomposedbyacomputer(3children,%correct)Scores 38,50,69Average 52Adultscoresonpoemstanzascomposedbyahuman(13adults,%correct) Levelofcomputerexperience
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 10/13
Little Moderate Professional AverageLittle 83 58,58,100 50,67 69Moderate 60,67,83 58,83 92 74Alot 25 25Average 73 72 59 68Childrensscoresonpoemstanzascomposedbyahuman(3children,%correct)Scores 33,42,50Average 42Adultscoresonallpoemstanzas(13adults,%correct) LevelofcomputerexperienceLevelofpoetryexperience Little Moderate Professional AverageLittle 68 50,64,86 57,71 66Moderate 55,61,71 61,68 82 66Alot 25 25Average 64 66 59 63Childrensscoresonallpoemstanzas(3children,%correct)Scores 39,43,61Average 48Numbersofrightandwronganswersforeachpoemstanza ComputerorhumanpoemstanzaPoemstanza No.right No.wrong1 9 7 computer3 11 5 computer4 8 8 computer6 9 7 computer8 11 5 computer11 11 5 computer13 8 8 computer
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 11/13
14 10 6 computer15 6 10 computer16 10 6 computer19 9 7 computer20 12 4 computer23 9 7 computer25 8 8 computer26 11 5 computer28 6 10 computerAverage 58% 42%2 9 7 human5 9 7 human7 9 7 human9 13 3 human10 8 8 human12 10 6 human17 9 7 human18 14 2 human21 11 5 human22 11 5 human24 11 5 human27 8 8 humanAverage 64% 36%Overallaverage 61% 39%
WehavenotyetreachedthepointatwhichcomputerscanevenarguablypasstheoriginallyproposedterminalinterviewtypeofTuringtest.Thistestrequiresacomputertomastertoomanyhighlevelcognitiveskillsinasinglesystemforthecomputeroftodaytosucceed.AsDanDennettpointsoutinhisarticle,theunadulteratedTuringtestisfarmoredifficultforacomputertopassthananymorerestrictedversion.Wehave,however,reachedthepointwherecomputerscansuccessfullyimitatehumanperformancewithinnarrowlyfocusedareasofhumanexpertise.Expertsystems,forexample,areabletoreplicatethedecisionmakingabilityofhumanprofessionalswithinanexpandingsetofhumandisciplines.Inatleastonecontrolledtrial,humanchessexpertswereunabletodistinguishthechessplayingstyleofmoresophisticatedcomputerchessplayersfromthatofhumans.Indeed,computerchessprogramsarenowabletodefeatalmost
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 12/13
allhumanplayers,withtheexceptionofasmallanddiminishingnumberofseniorchessmasters.MusiccomposedbycomputerisbecomingincreasinglysuccessfulinpassingtheTuringtestofbelievability.TheeraofcomputersuccessinawiderangeofdomainspecificTuringtestsisarriving.
Note1.Fourhumanpoetswereused:threefamouspoets(PercyByssheShelley,T.S.Eliot,andWilliamCarlosWilliams)andoneobscurepoet(RaymondKurzweil).Inthecaseofthefamoushumanpoets,stanzaswereselectedfromtheirmostfamouspublishedwork.Inallcases,thestanzasselecteddidnotrequireadjacentstanzastomakethematicorsyntacticsense.ThecomputerstanzaswerewrittenbytheKurzweilCyberneticPoetafterithadreadpoemswrittenbythesesamehumanauthors.Theanswersareasfollows:
Poemstanza1writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyWilliamCarlosWilliams
Poemstanza2writtenbyWilliamCarlosWilliams
Poemstanza3writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyPercyByssheShelley
Poemstanza4writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.EliotandPercyByssheShelly
Poemstanza5writtenbyRaymondKurzweil
Poemstanza6writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.Eliot,RaymondKurzweil,PercyByssheShelley,andWilliamCarlosWilliams
Poemstanza7writtenbyT.S.Eliot
Poemstanza8writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandT.S.Eliot
Poemstanza9writtenbyWilliamCarlosWilliams
Poemstanza10writtenbyRaymondKurzweil
Poemstanza11writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandT.S.Eliot
Poemstanza12writtenbyWilliamCarlosWilliams
Poemstanza13writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyPercyByssheShelley
-
6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI
http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 13/13
Poemstanza14writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.EliotandPercyByssheShelley
Poemstanza15writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandWilliamCarlosWilliams
Poemstanza16writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.Eliot,RaymondKurzweil,PercyByssheShelley,andWilliamCarlosWilliams
Poemstanza17writtenbyWilliamCarlosWilliams
Poemstanza18writtenbyRaymondKurzweil
Poemstanza19writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.Eliot,RaymondKurzweil,PercyByssheShelley,andWilliamCarlosWilliams
Poemstanza20writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandT.S.Eliot
Poemstanza21writtenbyT.S.Eliot
Poemstanza22writtenbyT.S.Eliot
Poemstanza23writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandWilliamCarlosWilliams
Poemstanza24writtenbyWilliamCarlosWilliams
Poemstanza25writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.Eliot,RaymondKurzweil,PercyByssheShelley,andWilliamCarlosWilliams
Poemstanza26writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandWilliamCarlosWilliams
Poemstanza27writtenbyPercyByssheShelley
