Possible DARPA IPTO Grand Challenge: Possible DARPA IPTO Grand Challenge: “WAIS Robot”“WAIS Robot”(WAIS(WAIS++ Robot?) Robot?)

Presented by: Dr. Selmer Bringsjord with Bettina Schimanski

The Challenge: The Challenge: WAIS RobotWAIS Robot

Autonomous artificially intelligent robots will display mastery of the Wechsler Adult Intelligence Scale (WAIS).

… + tests of creativity and mechanical ability…?

Intelligence Tests: Narrow vs. BroadIntelligence Tests: Narrow vs. BroadSpearman’sview of intelligence

Thurstone’s view ofintelligence

Narrow Test: RPMNarrow Test: RPM

=========== start of search ===========given clause #1: (wt=2) 10 [] R1(a11).given clause #2: (wt=2) 11 [] R1(a12).given clause #3: (wt=2) 12 [] R1(a13)....given clause #4: (wt=2) 13 [] R2(a21).given clause #278: (wt=16) 287 [para_into,64.3.1,3.3.1] R2(x)| -R3(a23)| -EmptyBar(y)| -R3(x)| -EmptyBar(x)| -T(a23)| -R3(y)| -T(y).given clause #279: (wt=16) 288 [para_into,65.3.1,8.3.1] R2(x)| -R3(a23)| -StripedBar(y)| -R3(x)| -StripedBar(x)| -EmptyBar(a23)| -R3(y)|-EmptyBar(y).Search stopped by max_seconds option.============ end of search ============

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

WAIS includes many sub-testsWAIS includes many sub-tests

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Comprehension sub-testComprehension sub-test

The Comprehension sub-test is so difficult it The Comprehension sub-test is so difficult it could be a motivator for the original CYC dreamcould be a motivator for the original CYC dream

-Deals with ordinary conversationDeals with ordinary conversation

Examples: Examples:

““Why is copper often used in making Why is copper often used in making electrical wires?”electrical wires?”

““What is the advantage of keeping What is the advantage of keeping money in a bank account?”money in a bank account?”

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Block Design sub-testBlock Design sub-test

-Arrange blocks toArrange blocks to

match a given samplematch a given sample

(Blocks courtesy of Binary Arts Corporation)

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Object Assembly sub-testObject Assembly sub-test

- Presented with several 2D puzzle-like pieces- Presented with several 2D puzzle-like pieces

Object AssemblyObject Assembly

Sample provided by Psychological Corporation (© 2004)Sample provided by Psychological Corporation (© 2004)

Broad Test: WAISBroad Test: WAISWechsler Adult Intelligent ScaleWechsler Adult Intelligent Scale

Picture Arrangement sub-testPicture Arrangement sub-test

- Deals with many mental facets- Deals with many mental facets

Picture ArrangementPicture Arrangement

WAIS Challenge Decidedly WAIS Challenge Decidedly Non - Ad HocNon - Ad Hoc

Integrated intelligence via Alan Newell’s third paradigmIntegrated intelligence via Alan Newell’s third paradigm ““You Can’t Play 20 Questions with Nature and Win” (1973)You Can’t Play 20 Questions with Nature and Win” (1973)

““We never seem in the experimental literature to put the We never seem in the experimental literature to put the results of all the experiments together”results of all the experiments together”

First Paradigm: Complete Processing ModelsFirst Paradigm: Complete Processing Models Second Paradigm: Analyze a Complex TaskSecond Paradigm: Analyze a Complex Task Third Paradigm: One Program for Many TasksThird Paradigm: One Program for Many Tasks

The same observation can be made today about The same observation can be made today about systems engineered in AI (Building Block Approach)systems engineered in AI (Building Block Approach) The idea is to use different subtests from certain chosen tests of The idea is to use different subtests from certain chosen tests of

mental and/or physical ability to create a composite artificial agent mental and/or physical ability to create a composite artificial agent for a specific application.for a specific application.

The Proposed RulesThe Proposed Rules The artificial agent must operate without any prior The artificial agent must operate without any prior

knowledge of actual test course (must handle any knowledge of actual test course (must handle any case in the general space)case in the general space)

The artificial agent will be administered the test as a The artificial agent will be administered the test as a human subject would:human subject would: Must physically manipulate objects (of many different shapes Must physically manipulate objects (of many different shapes

and sizes but weighing less than 4 lbs. each)and sizes but weighing less than 4 lbs. each) Must be able to take spoken directionsMust be able to take spoken directions Must be able to respond to questions of varied topicsMust be able to respond to questions of varied topics Must have a vision componentMust have a vision component Must be subject to the same time limitationsMust be subject to the same time limitations

The artificial agent operates autonomously once the The artificial agent operates autonomously once the test has started until completiontest has started until completion

Training & Test MaterialsTraining & Test Materials

Some homegrown test samples/examplesSome homegrown test samples/examples

MilestonesMilestones

Year 1: Feasibility demonstration

Complete individual sub-tests at different year cutoffs:

Year 2: Digit Symbol, Digit Span, Vocabulary, Arithmetic, Similarities Year 3: Block Design, Object Assembly Year 5: Picture Completion Year 7: Information. Comprehension Year 9: Picture Arrangement

Year 10: Complete all sub-tests within time limit given for humans (in terms of physical manipulation, language parsing, visual deciphering, etc.)

Milestones: Permutation 1Milestones: Permutation 1 Year 1: Feasibility demonstration

Complete individual sub-tests at different year cutoffs:

Year 2: Digit Symbol, Digit Span, Vocabulary, Arithmetic, Similarities Year 3: Block Design, Object Assembly Year 5: Picture Completion Year 7: Picture Arrangement Year 9: Information, Comprehension

Year 10: Complete all sub-tests within time limit given for humans (in terms of physical manipulation, language parsing, visual deciphering, etc.)

LimitationsLimitations

The WAIS does not test The WAIS does not test

generative ability muchgenerative ability much Short Short Story Game (SShort Short Story Game (S33G)G) S. Bringsjord (1998). "Chess is Too Easy", Techn

ology Review, 101, 2, 23-28.

What about locomotion etc.?What about locomotion etc.?

Background ReferencesBackground References S. Bringsjord and B. Schimanski (2003). "What

is Artificial Intelligence? Psychometric AI as an Answer", Proceedings of the 18th

International Joint Conference on Artificial Intelligence (IJCAI-03), Morgan Kaufm

ann, San Francisco, CA, 887-893.

S. Bringsjord and B. Schimanski (2004). "Pulling It All Together via Psychometric AI", Achieving Human-Level Intelligence Through Integrated Systems and Research: Technical Report FS-04-01, AAAI Press, Menlo Park, CA, 9-16.