Download - The Age of Intelligent Machines _ a (Kind of) Turing Test _ Kurzweilai-20

6/20/2015 THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest|KurzweilAI

http://www.kurzweilai.net/theageofintelligentmachinesakindofturingtest 1/13

KurzweilAI|AcceleratingIntelligence.ebook:TheAgeofIntelligentMachines

THEAGEOFINTELLIGENTMACHINES|A(KindOf)TuringTest

September24,2001

author|RayKurzweil

Noideasbutinthings.

WilliamCarlosWilliams

Asdiscussedinseveralofthecontributedarticlesinthisbook,theTuringtestwasdevisedbyAlanTuringasawayofcertifyingmachineintelligence.Turingdescribedasituationinwhichahumanjudgecommunicateswithbothacomputerandahumanusingacomputerterminal.Thejudgestaskistodeterminewhichiswhich.Thejudgecannotseethecomputerorthehumanandmustmakehisorherdeterminationbyinterviewingboth.Thecomputerattemptstotrickthejudgeintoselectingitasthehuman.

TheessenceoftheTuringTestisthatthecomputerattemptstoactlikeahumanwithinthecontextofaninterviewoverterminallines.AnarrowerconceptofaTuringtestisforacomputertosuccessfullyimitateahumanwithinaparticulardomainofhumanintelligence.WemightcallthesedomainspecificTuringtests.OnesuchdomainspecificTuringtest,basedonacomputersabilitytowritepoetry,ispresentedhere.

TheKurzweilCyberneticPoetisacomputerprogram(writtenbytheauthor)andprovidedwithaninputfileofpoemswrittenbyahumanauthororauthors.Theprogramanalyzesthesepoemsandcreatesawordsequencemodelbasedonthepoemsithasjustread.Itthenwritesoriginalstanzasofpoetryusingthemodelithascreated.SomeofthefollowingstanzasofpoetrywerewrittenbytheKurzweilCyberneticPoet.Somewerewrittenbyhumanauthors(infactthesamehumanauthorsthatwerereadandanalyzedbytheKurzweilCyberneticPoet).Seeifyoucantellwhicharewhich.Onapieceofpaper,writedownthenumbers1through28.ThenputaCbythatnumberifyoubelievethatthecorrespondingstanzawaswrittenbythecomputer.PutanHifyoubelievethestanzawaswrittenbyahumanpoet.Theanswersareinafootnote.1Followingtheanswersaretheresultsobtainedwhenthetestwastakenby16human(bothadultandchild)judges.

1.isbeautyitself

thattheywerewalkingthere.Allalongthenewworldnaked,



cold,familiarwind

2.Pinkconfusedwithwhite

flowersandflowersreversed

takeandspilltheshadedflame

dartingitback

intothelampshorn

3.Thewindsoftheoozywoodswhichwear

theocean,withazuremossandflowers

Sosweet,thepurpleeven

Isleepinthearrows

Ofthedomeofdeath.

4.0thou,

WhomovedamongsomefierceMaenad,evenamongnoise

andblue

Betweenthebonessang,scatteredandthesilentseas.

5.Sheeyesmewithaningrowneye,

intherhythmofteacuptapping

thinksofsweepingawaycrumbs

6.AtsixIcannotpray:

Prayforlovers,



throughnarrowstreets

Andpraytofly

ButtheVirginintheirdarkwintrybed

7.Whatseaswhatshoreswhatgraniteislandstowardmytimbers

Andwoodthrushcallingthroughthefog

Mydaughter.

8.Imaginenowatreeinwhitesailsstillwhirled

Abouttheleaves

willbeofsilences

Calmandangels

9.andthesun,dippingintotheavenues

streakingthetopsof

theirregularredhouselets,

and

thegayshadowsdroppinganddropping.

10.Themorningandalready

aperfectifslightlypaled

oldparkturnedwithyoungwomen

seizedinamber.

11.Interestingbook?



shesits

dancingbytheelectrictypewriter,

bloodlessrevolutionofmeats

stringsofuse,

Politic,cautious,andthefact

sheisthefact

sheiscallingthemall

Thechildrenathisfeet

heisalwaystime

Torollitwasdark,

damp,jagged,likethevoice

Becauseofloveends.

12.Menwithpickedvoiceschantthenames

ofcitiesinahugegallery:promises

thatpullthroughdescendingstairways

toadeeprumbling.

13.Wherewerethou,sadHour,selectedfromwhoseraceis

guidingme,

LuredbytheloveofAutumnsbeing,

Thou,fromheavenisgone,wherewaslornUrania



Whenrockedtoflywiththeeinherclarionoerthearmsofdeath.

14.LadyofAutumnsbeing,

Thou,fromtheday,havingtocare

Teachusnowthoroughlysmallandcreate,

Andthenpresume?

Andthis,andme,

Andplaceoftheunspokenword,theunreadvisioninBaiaesbay,

AndtheposterityofMichelangelo.

15.Iamlonely,lonely.

Islapananswermyself

shehidesdeepwithinher

yetplays

Milkless

16.0myshoulders,flanks,buttocks

againsttrespassers,

againstthieves,

storms,sun,fire,

againstthieves,

storms,sun,fire,

againstflies,againstweeds,stormtides,



neighbors,weaselsthatwaken

Thesilentseas.

17.thedays,lockedineachothersarms,

seemstill

sothatsquirrelsandcoloredbirds

goaboutateaseover

thebranchesandthroughtheair.

18.Iamwatchingantsdigtunnelsandburythemselves

theygowithoutwaterorlove

19.Ladyissick,

perhapsvomiting,

perhapslaboring

totheusualreign

20.Rainissweet,brownhair

Distraction,musicinpassageways.

Sixoclock.

Thetime.Redeem

Theworldandwaking,wearing

21.Wipeyourhandacrossyourmouth,andlaugh

Theworldsrevolvelikeancientwomen



Gatheringfuelinvacantlots.

22.Ishouldhavebeenapairofraggedclaws

Scuttlingacrossthefloorsofsilentseas.

23.patchesofall

savebeauty

therigidwheeltracks.

Theroundsun

thebed.

Shesmiles,Yes

youpleasefirst

thenstays

withherselfalone

andthendividingandover

andsplashedandafteryouare

listeninginhereyes

24.Allalongtheroadthereddish

purplish,forked,upstanding,twiggy

stuffofbushesandsmalltrees

withdead,brownleavesunderthem

leaflessvines



25.Prayforthosewhoarebranchesonforever

26.Likeasodofwar

housesofsmall

whitecurtains

Smellofshimmering

ashwhite,

anaxe

27.Byactionorbysuffering,andwhosehour

Wasdrainedtoitslastsandinwealorwoe,

Sothatthetrunksurvivedbothfruitorflower

28.isasteadyburning

theroadthebattlesfury

cloudsandashandwaning

sendingout

youngpeople,

Theabove28questionpoeticTuringtestwasadministeredto16humanjudgeswithvaryingdegreesofcomputerandpoetryexperienceandknowledge.The13adultjudgesscoredanaverageof59percentcorrectinidentifyingthecomputerpoemstanzas,68percentcorrectinidentifyingthehumanpoemstanzas,and63percentcorrectoverall.Thethreechildjudgesscoredanaverageof52percentcorrectinidentifyingthecomputerpoemstanzas,42percentcorrectinidentifyingthehumanpoemstanzas,and48percentcorrectoverall.

Thechartsshowtheactualscoresobtainedbythe16humanjudgesasbrokendownbyadult/child,computerexperience,andpoetryexperience.Ascanbeseenfromthecharts,therewerenotrendsbasedonlevelofcomputerexperienceorpoetryexperienceclearlydiscerniblefromthislimitedsample.Theadultsdidscoresomewhatbetterthanthechildren.Thechildrenscoredessentiallyatchancelevel(approximately50percent)andtheadultsachievingslightlybetterthanchance.



Thenextchartshowsthenumberofcorrectandincorrectanswersforeachofthe28poemsorstanzas.Whiletheadultjudgesscoredsomewhatbetterthanchance(63percent),theiranswerswerefarfromperfect.Thecomputerpoetwasabletotrickthehumanjudgesmuchofthetime.Someofthecomputerpoems(numbers15and28,forexample)wereparticularlysuccessfulintrickingthejudges.

WecanconcludethatthisdomainspecificTuringtesthasachievedsomelevelofsuccessintrickinghumanjudgesinitspoetrywritingability.Amoredifficultproblemthanwritingstanzasofpoetryiswritingcompletepoemsthatmakethematic,syntactic,andpoeticsenseacrossmultiplestanzas.AfutureversionoftheKurzweilCyberneticPoetiscontemplatedthatattemptsthismoredifficulttask.Tobesuccessful,themodelscreatedbytheCyberneticPoetwillrequirearicherunderstandingofthesyntacticandpoeticfunctionofeachword.

EventheoriginallyproposedTuringtestinvolvingterminalinterviewsisnotablyimpreciseindeterminingwhenthecomputerhasbeensuccessfulinimitatingahuman.Howmanyjudgesneedtobefooled?Atwhatscoredoweconsiderthehumanjudgestohavebeenfooled?Howsophisticateddothejudgesneedtobe?Howsophisticated(orunsophisticated)doesthehumanfoilneedtobe?Howmuchtimedothejudgeshavetomaketheirdetermination?ThesearebutafewofthemanyquestionssurroundingtheTuringtest.(ThearticleACoffeehouseConversationontheTuringTestbyDouglasHofstadterinchapter2providesanentertainingdiscussionofsomeoftheseissues).ItisclearthattheeraofcomputerspassingtheTuringtestwillnothappensuddenly.OncecomputersstarttoarguablypasstheTuringtest,thevalidityofthetestsandthetestingprocedureswillundoubtedlybedebated.ThesamecanbesaidforthenarrowerdomainspecificTuringtests.

Adultscoresonpoemstanzascomposedbyacomputer(13adults,%correct) LevelofComputerexperienceLevelofpoetryexperience

Little Moderate Professional AverageLittle 56 44,69,75 63,75 64Moderate 50,56,63 56,63 75 61Alot 25 25Average 56 61 59 59Childrensscoresonpoemstanzascomposedbyacomputer(3children,%correct)Scores 38,50,69Average 52Adultscoresonpoemstanzascomposedbyahuman(13adults,%correct) Levelofcomputerexperience



Little Moderate Professional AverageLittle 83 58,58,100 50,67 69Moderate 60,67,83 58,83 92 74Alot 25 25Average 73 72 59 68Childrensscoresonpoemstanzascomposedbyahuman(3children,%correct)Scores 33,42,50Average 42Adultscoresonallpoemstanzas(13adults,%correct) LevelofcomputerexperienceLevelofpoetryexperience Little Moderate Professional AverageLittle 68 50,64,86 57,71 66Moderate 55,61,71 61,68 82 66Alot 25 25Average 64 66 59 63Childrensscoresonallpoemstanzas(3children,%correct)Scores 39,43,61Average 48Numbersofrightandwronganswersforeachpoemstanza ComputerorhumanpoemstanzaPoemstanza No.right No.wrong1 9 7 computer3 11 5 computer4 8 8 computer6 9 7 computer8 11 5 computer11 11 5 computer13 8 8 computer



14 10 6 computer15 6 10 computer16 10 6 computer19 9 7 computer20 12 4 computer23 9 7 computer25 8 8 computer26 11 5 computer28 6 10 computerAverage 58% 42%2 9 7 human5 9 7 human7 9 7 human9 13 3 human10 8 8 human12 10 6 human17 9 7 human18 14 2 human21 11 5 human22 11 5 human24 11 5 human27 8 8 humanAverage 64% 36%Overallaverage 61% 39%

WehavenotyetreachedthepointatwhichcomputerscanevenarguablypasstheoriginallyproposedterminalinterviewtypeofTuringtest.Thistestrequiresacomputertomastertoomanyhighlevelcognitiveskillsinasinglesystemforthecomputeroftodaytosucceed.AsDanDennettpointsoutinhisarticle,theunadulteratedTuringtestisfarmoredifficultforacomputertopassthananymorerestrictedversion.Wehave,however,reachedthepointwherecomputerscansuccessfullyimitatehumanperformancewithinnarrowlyfocusedareasofhumanexpertise.Expertsystems,forexample,areabletoreplicatethedecisionmakingabilityofhumanprofessionalswithinanexpandingsetofhumandisciplines.Inatleastonecontrolledtrial,humanchessexpertswereunabletodistinguishthechessplayingstyleofmoresophisticatedcomputerchessplayersfromthatofhumans.Indeed,computerchessprogramsarenowabletodefeatalmost



allhumanplayers,withtheexceptionofasmallanddiminishingnumberofseniorchessmasters.MusiccomposedbycomputerisbecomingincreasinglysuccessfulinpassingtheTuringtestofbelievability.TheeraofcomputersuccessinawiderangeofdomainspecificTuringtestsisarriving.

Note1.Fourhumanpoetswereused:threefamouspoets(PercyByssheShelley,T.S.Eliot,andWilliamCarlosWilliams)andoneobscurepoet(RaymondKurzweil).Inthecaseofthefamoushumanpoets,stanzaswereselectedfromtheirmostfamouspublishedwork.Inallcases,thestanzasselecteddidnotrequireadjacentstanzastomakethematicorsyntacticsense.ThecomputerstanzaswerewrittenbytheKurzweilCyberneticPoetafterithadreadpoemswrittenbythesesamehumanauthors.Theanswersareasfollows:

Poemstanza1writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyWilliamCarlosWilliams

Poemstanza2writtenbyWilliamCarlosWilliams

Poemstanza3writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyPercyByssheShelley

Poemstanza4writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.EliotandPercyByssheShelly

Poemstanza5writtenbyRaymondKurzweil

Poemstanza6writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.Eliot,RaymondKurzweil,PercyByssheShelley,andWilliamCarlosWilliams

Poemstanza7writtenbyT.S.Eliot

Poemstanza8writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandT.S.Eliot





Poemstanza13writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyPercyByssheShelley



Poemstanza14writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyT.S.EliotandPercyByssheShelley

Poemstanza15writtenbytheKurzweilCyberneticPoetafterreadingpoemsbyRaymondKurzweilandWilliamCarlosWilliams












Poemstanza27writtenbyPercyByssheShelley