From the Ground Up: A.I. Architecture and Design

Patterns Dr Brett Laming

Rockstar Leeds

Contents

• Basic Principles• Think/Act - The I/O Divide• An Algorithmic Approach• A Functional Approach• Movement• Animation• Conclusion

Basic Principles

Why Do We Care?• It’s a Competitive

Market…– Higher Expectations– Tighter Schedules– Multiple Platforms– Simultaneous Titles

• …But Patterns Are Everywhere– Identify, generalise,

reuse, evolve Reliability Production Speed More Fun Stuff

Inspirations

Marvin Minsky Different representations

for different viewsNoam Chomsky

Hierarchical decomposition

Structure vs Meaning“Colorless green ideas

sleep furiously”

Daniel DennetBehaviour can be

viewed at the physical, design and intentional

levels

David MarrComputational, algorithmic and

implementationalChris HeckerStyle vs Structure [2008]

What is the textured triangle of A.I.

Craig ReynoldsSimple rules

Complex behaviour

Damian IslaCognitive Maps

Spatial RelationsSemantics

What Am I Looking For?Algorithm

s

Best Practice

Hierarchy

Reusability

Commutability

ConceptsCom

ponents

How Do I Find Them?

• Observation– How might it work?

• Introspection– What would I do?

• Generalisation– I’ve done this before– They do the same

• Bad Experience– Lets not do that

again

• Background– I studied this once?– Could I apply this?

Best Practice

Best Practice• Prototype new ideas where possible

– Get visual and design direction• Mock-ups

– Prove (or disprove) the concept• Quick and dirty programming

• Play to peoples strengths– Maths Physics Guys– Navigation Mesh Collision Guys

Best Practice• Program Defensively

– Assert and Unit Test– Automated Scripts as soak tests– One co-ordinate system and S.I. units

• Maximise Workflow– Cater tools to their needs– Put new functionality on bypass

• Think of the man-hour cost!– Minimise potential for human error

Best Practice• Build A Debugging Suite

– Instant Pause– Flyable Camera– Layered Information– Action Histories

• Maintain Player Immersion– A.I. should not be too bad or too good– Limit ourselves to what the player would know– Constrain them to the same actions

The Algorithmic Approach

• Key tenants– Use only what

we might know– Mimic the player

The Think-Act Loop

Sensory Receptors

Brain

Controller

Game

Player

Sensory Data

Think

Controller

Act

A.I.

Sensory Data

Think

Controller

Act

Sensory Input

Sensory Data• Detail level

– Depends on genre and perceived communication• Thief vs Medieval Total War II

• General Model– Visual component an arc– Auditory component a radius

• Auditory targets less official– Occlusion too expensive?

• Shoot a weapon to get info– Theorise using ghost images

Blackboards• Used To Share Information

– Static blackboard stores defined types of info– Dynamic blackboard stores arbitrary data

• Agents write to the board– Generally read it as well

Agent A

Agent B

{10,20,15}

{-30,20,13}

{-10,15,12}

{17,11,5}

Scout {3, 17, 10}Cover {17, 11,5}

cSensorCone

Think

Controller

Act

Control Output

Virtual Controller (yoke)

• Purpose– Carries control instructions– Provides a strict I/O Divide

• Notice the const correctness– Unifies player and A.I.

• Control mapping for the player• A.I. fills it in from Think()

• Key Properties– Never stored in it’s entirety– Created on the stack– Lifetime of a single Process()

Process()

Think(yoke&) const

Controller

Act(const yoke&)

cVirtualYoke yoke

… So Actually

Process()

Think(yoke&) const

Act(const yoke&)

cVirtualYoke yoke

cSensorCone

cBlackboard

Virtual Yoke

• Composite– Collection of smaller yokes– Allows selective storage

cVirtualYoke

cLocomotionYoke mLocomotion;cWeaponYoke mWeapons;cAnimYoke mAnimation;

cLocomotiveYoke

fp32 mGas;fp32 mSteering;fp32 mPitch;

cEntity *mpDontAvoid;

• Context Based Controller Input

• Logic Flow Control

cWeaponYoke

enum eFire{ F_PRESSED, F_WHEN_IN_CONE, F_WHEN_LOCKED};

eFire mFireButton;

• Conditional signals

Taking Action

Process()

Think(yoke&) const

Act(const yoke&)

cVirtualYoke yoke

cSensorCone

cBlackboard

Act

• Applies Yoke Commands– Composite yokes

Subsystems• Object Models

– Supply common interfaces• Could be a turret mounted

weapon or my pistol.• Could be driving a car, a plane,

a boat or myself!

Act()

yoke

iWeaponMgr*

iLocomotive*

Object Model

• Self Contained– Instructions for Think– Actions for Act – Commutable

• Plug and Play• Downloadable

Content– Maybe broadcast use

• Think Sims 2!

iLocomotive

// Fills in yokeComputeMotion(const cTarget &,

cLocomotiveYoke&)

// Computes forcesApplyMotion(const cTarget &, const cLocomotiveYoke&,

v3 &force, v3 &torque)

More Coolness

ApplyMotion(const cTarget& targ, cLocomotiveYoke& yoke)

{v3 force(Zero);v3 torque(Zero);if (Drivable()){

Drivable()->ApplyMotion( targ, yoke, force, torque);

}if (Floatable()){

Floatable()->ApplyMotion( targ, yoke, force, torque);

}ApplyForceTorque(force, torque);

}

Compute

Apply

Compute

Apply

Compute

Apply

Compute

Apply

Frame Time

Process()

Think(yoke&, dt) const

Act(const yoke&, dt)

cVirtualYoke yoke

cSensorCone

Update(dt)

cBlackboard

dt()

dt()• Work into everything

– Including fixed time steps– No more s+=v, s+=v*dt()

• Benefits– Integration

• Implicit forward Euler– Rough Smoothing– Closed Feedback– Pause dt=0– Level Of Detail dt=2dt– Special Effects

dt()

smoothing

feedback control

The Functional Approach

Problem Domain

• Examine the Terminology– Feeling, Knowledge, Goals, Beliefs, Needs

• Examine the Concepts– Decisions, Facts, Uncertainty, Exploration, Verb-Noun

Actions, Repetition, Sequencing

Needs a problem

What Are We Doing?• Goal based reasoning

– Working to solve a goal– Thinking about and realising smaller tasks– Taking a hierarchical approach– Using a limited number of short verb-noun pairings to

form a plan• We’ve seen this before

– An old pattern• Colossal Cave Adventure and MUDs• Verb-noun actions like “get axe, wield axe” separated by

movement “n, e, e, s, e”– We use a container object model

Applying The Pattern

Know of “Mine, Smithy”1. Goto “Mine”Know of “Wall”2. Get Ore From “Wall”3. Goto “Smithy”Know of “Door”4. Use “Door”

5. Play “Open Door”6. Warp InsideKnow of “Owner”, “Forge”7. Play “Close Door”

8. Goto “Owner” 9. Put 10 gp In “Owner”10. Get Time From “Owner”11. Goto “Forge”12. Put Time In “Forge”13. Put Iron In “Forge”14. Use “Forge”15. Use Enchant16. Use “Forge”17. Get Sword From “Anvil”

Taking the computational to the algorithmic

Easy Questions

• Why did I choose to do this again?– Because we were driven to it by personality

and need• What happens when I get the treasure?

– I’ll probably choose to do something else depending on my mood

• Why stop at get iron?– Because its reached the atomic level - there

are no more questions, just results

Putting It Together

AmbientController“Get treasure”

Planning“Get magic sword”

Explicit Order

“Script here,Please greet

the player”

Explicit Orders

• Script commands– Script on A.I.

• Autopilot– Player on Player

• Player instructions– Player on A.I.

• Squad Commander – A.I. on A.I.

AmbientController“Get treasure”

Planning“Get magic sword”

Explicit Order

“Script here,Please greet

the player”

Scripting Notes• Don’t mix styles

– Script has immediate control– Script waits for an opportunity

• Keep common properties separate– No sharing memory locations

• A script global population density• A code global population density

– Maintain a set order of calculation• Generally consistent with style

Ambient Controller

• Generates sensible actions autonomously– Maybe Idle– Maybe Full Daily Routine

• Daily Routines– Character properties– Needs/Drives– Scheduling– Time of day nice emergent behaviour

AmbientController“Get treasure”

Planning“Get magic sword”

Explicit Order

“Script here,Please greet

the player”

Daily Routine

Sleep

Goto Work

Work

Leave Work

Relax

Go Home

Drive ModelTime Of Day

Schedule

Housework 5%

Tavern 50%

Brothel 30%

Shopping 10%

Study 5%

Character

Hunger Libido

dt()

Planning

AmbientController“Get treasure”

Planning“Get magic sword”

Explicit Order

“Script here,Please greet

the player”

Goto Position“Goto Forge”

Perform Action“Use Forge”

Planning

Continuous bar dt()Defines Think()

Plan Components

• Play “Animation”– Waits on dt()

• Use “Object”– Object model again– Broadcast actions.– Change world based on

state– May wait on dt()

• Get/Take “Object” From “Container”– Primary world manipulation– Contents determine state

• Goto “Location”– Waits on dt()– Key A.I. output– Complex– Warrants special attention

later

Search Based Planning

• Traditional academic approach– See STRIPS, Hierarchical Task Networks, Bratko

• The Good– Mimics our introspective reasoning– Seeks to fully realise a plan to the goal

• Directed search for optimal solutions• Post processing even more so

• The Bad– Knowledge representation

• Scalability - difficult for video games• Lots of storage

Procedural Planning• Industry preferred approach• Hierarchical• Easy to comprehend• Limited Language

– Goals– Sub-goals– Conditions– Actions

• Transitions– Sequential– Decision Based

• Powerful Results

Get Object

Short of money?

Make

Thief skills?

Steal

Buy

Get OreGoto Mine

Procedural Planning - Issues• Competing children

– We have to make a best guess from the options

– A* might help • But we could still end up with case of a basic

sword being bought but not affording the forge.

• Incomplete plan means no post process– Not good for player supporting A.I.– More action for generic A.I.

The Curve Ball• Task Interruption

– I’m returning to my gang hideout– I see an enemy

• I engage the enemy–I roll out of the way of a car–I recover to my feet

• I re-engage the enemy– I continue to return to my gang hideout

• … Is A Key Requirement of our A.I.

Finite State Machines

• No plan history No idea of contextNo generalised exit.Hideous state history

workarounds• Don’t scale well

– Many transitions

Death

Look At

Gain Range

Attack

Kill ExamineDeath

Look At

Gain Range

Attack

Kill Examine

• HFSM came along– Eased transitions– But history still an issue

Behaviour Trees

Goto Source

Get Material

Take Material

Create Buy

Steal

Get Object

Crime Allowed?

Lowmoney?

Okmoney?

Thief Class?

Sequencing

Selection

PreconditionsActions

Decorators

Modelling Interruption?

Goto Location

KillEnemy

Return Home

Locate Hideout

GainRange

FireWeapon

Ambient Behaviour

Parallel?

Behaviour Trees• Simple and powerful

– Limited vocabulary– Most situations handled

• Highly flexible– Plug and play– Customisable– Nice design tools– Handy child evaluation

• Lends itself to directed decision making

• Issues– Interruption handling

• Where to return

– Amount of flexibility• Trees get complicated

MARPO

Goto Source

Get Material

Take Material

Create Buy Steal

Get Object Not enough money? Money?

Thief class?Thieving allowed?

ReactiveLong term

MARPO

Get Object

Create

Get Material

Take Material

Immediate

ReactiveLong term

MARPO

Get Object

Take Material

Kill Enemy

Gain Distance

Immediate

Get Material

Create

ReactiveLong term

MARPO

Get Object Kill Enemy

Immediate

Roll

Gain Distance

• Behavioural Bamboo Forest– Stack based tasks

• Suppression model– Multiple threads of execution

• Keeps only one stack in memory down to current task– Decision logic lies with the parent

• Higher level parameterised building blocks– Authoring is by script not tool.

• Winding Ability– Allows auto-recovery from any state– Respects script orders on an immediate basis

MARPO

Movement

Goto Position

• What Is Position?– A world co-ordinate– An entity– A navigation point– An offset off an entity– A radius off an offset,

off an entity• Still World Positions!

– So generalise

cTarget

void Set(…);void SetArrivalConditions(…);

v3 WorldPos() const;v3 InterceptPos(…) const;bool HasArrived(…) const;

• Lots for free– Entity intercept– Completion checks

The Problem

• Dynamic obstacles• … on the move• Terrain type

• Usable surfaces

• Static geometry

Stage 1 - Navigation

• Navigation Mesh– Industry standard [Tozour, 2008]– Handles static geometry

• Searching with A*– Optimise search space, not A*

• Improve Data Format– Make it Hierarchical– Allow for reference spaces [Isla, 2005]

Navigation

• Quick To Search Live Updates

• Zones Spatial Reasoning [Isla, 2005]

cNavPos

Components

cNavId

mPolyId;mPolySource;

cNavPoly

… indices… edge info

cNavId mParentcNavId mFirstChild

mChildCount

cPolyCoords

… local co-ords

cNavigatorPos()FinalTarget()CurrentTarget()HasArrived()Update( cTarget )

cNavPos mFinalcNavPos mCurrentcNavPos mPos

Stage 2 - Optimisation

• String Pulling– Bevel uncrossable

edges to a radius– Tighten points

Stage 3 - Dynamic Avoidance

• At rest– Mesh binding

• On move– Intercept calculation

Stage 4 - Locomotion

• Differing approach– Vehicle– Humanoid

• Can we generalise?

Locomotion

iLocomotion

ComputeMotion(…)ApplyMotion(…)

• Generalised by iLocomotion• Same interface• Different yoke instructions• Vehicles

– Vehicles supply gas, steering– Difficulty is in mapping target to gas and steering

• Actors– Actors supply ideal position, velocity, direction– Difficulty is in animation to hit position.

Locomotion - Vehicles

TargetPosition

Top Speed Speed For Deceleration

Speed For Steering

Min

Speed to GasMaths

PID ControllerYoke

YokeAngle To Steering

MathsPID Controller

Some Hints

a d

vmax

u vs1 s2

Speed For Deceleration

• Equations of Motion

Speed to GasAngle To Steering

• PID • Response Curves

Manual Shift

Animation

Locomotion - Character

• Animation Position• Challenges

– Sensible manoeuvre choice

• Animation Graphs• Navigation Mark Up

– Natural fluidity• Foot positioning• Hand positioning

Idle

Walk

Run

Crouch

Crouch Walk

Crouch Run

Goal Based

• Directed Search of Animation Graph– Incorporates interesting A.I. along the way– Think Gears of War, A*

• Interested Champandard• Personal Issues

– Need to find a full animation plan to goal• This would need to sit with path finding• Makes it more expensive

– Painful Heuristic Balancing• Switching to the time domain probably partially solves it• But how do we prevent repeated actions

“Carrot On A Stick” Method

• Keep iLocomotion interface• ComputeMotion() computes

velocity• ApplyMotion() animates for

velocity • dt() helps minimise error• Want to use a cover point?

– Explicit decision a new target

iLocomotion

ComputeMotion(…)ApplyMotion(…)

• Given the animation properties

• We can solve by mathematics

Cover

Under fire?

Find cover?

Route to cover

Kill

Nav. m

esh

Goto

Set piece

“Carrot On A Stick” Method

Anchor Point

“Carrot On A Stick” Method

s

Direction for Animation off=ƒ(,s)

• Offset based on , s• Changes our target

– Hierarchical changes

• Changing over dt• Smoothes our arrival

– Think high jump!– Never perfect– But mistakes made

are human– Constrained by

navigation mesh

“Carrot On A Stick” Method

su

V = 0?

• Equations of Motion– They’re back!

• Give us our aim velocity vaim

• What about a and d?

vmaxad

u vs

vaim

dt

• Animations– distance (s) in time (t) an average velocity– Assume velocity bands– Look up v for animation– Dead zones indirectly

determine blending• a and d now character

properties• Actively correct for v

– Use tricks like light I.K.• Implicitly correct by dt()

v

t

Crouch run

Crouch walk

v

Crouch walk

Crouch run

“Carrot On A Stick” Method

“Carrot On A Stick” Method• Animation graph

– Used for information query only• Locomotive cycle driven by vaim

• Free things!– Foot positioning

• Obtained through cycle alteration and dt()– Blending– Character movement properties

• Obtained through a and d– Swim, crawl, crouch, walk

• Same procedure, different velocity bands– Human like arrival mistakes

Conclusion

Conclusion

– Best Practices– Strict interfaces– Multiple takes on

problems– Concepts– Commutability

– Components– Algorithms– Hierarchy– Pattern Recognition– Human Sciences

Good reusable AI is about

Conclusion

• Style vs Structure - Hecker 2008– “Texture Mapped A.I. Triangle”– Style

• No idea where it is – annoyingly– Structure

• Not one single triangle• But many triangle like pieces• Represented differently• But all essentially the same piece

References• Key References

– Laming, B. [04] “The Art of Surviving A Simulation Title”, A.I. Wisdom 2, Charles River Media – Laming, B. [08] “The MARPO Methodology: Planning And Orders”, A.I. Wisdom 4, Charles River Media– Isla, D. [05], “Dude, where’s my Warthog?”, www.aiide.org/aiide2005/talks/isla.ppt, 13.3.08– Tozour, P. [08] “Fixing Pathfinding Once and For All”, http://www.ai-blog.net/archives/000152.html, 13.3.08– Champandard, A., http://aigamedev.com, 13.3.08– A.I. Wisdom Books in general

• Introduction– Reynolds, C. [87] “Boids”, http://www.red3d.com/cwr/boids/, 13.3.08

• Intercept Calculation and Dynamic Avoidance– Stein, N. [02] “Intercepting a Ball”, A.I. Wisdom 1, Charles River Media. [HINT ]– Tozour, P. [02] “Building a Near-Optimal Navigation Mesh”, A.I. Wisdom 1, Charles River Media

• Best Practice– Tozour, P. [02] “Building an AI Diagnostic Toolkit”, A.I. Wisdom 1, Charles River Media

• Sensory and Blackboards– Isla, D. and Blumberg, B. [02] “Blackboard Architectures”, A.I. Wisdom 1, Charles River Media

• Planning– Champandard, A [08], “Getting Started With Decision Making and Control Systems”, A.I. Wisdom 4, Charles River Media– Champandard, A., “Understanding Behaviour Trees”, http://aigamedev.com/hierarchical-logic/bt-overview, 13.3.08– Yiskis, E. [04] “A Subsumption Architecture for Character-Based Games”, A.I.Wisdom 2, Charles River Media

• Navigation– Numerous references, almost all A.I. Wisdom books.

• Locomotion – Vehicle– Alexander, B. [02] “The Beauty of Response Curves”, A.I. Wisdom 1, Charles River Media– Forrester, E. [04] “Intelligent Steering Using PID Controllers”, A.I. Wisdom 2, Charles River Media

• Conclusion– Hecker, C. [09] “Structure vs Style”, http://chrishecker.com/Structure_vs_Style, 13.3.08